-- Logs begin at Fri 2022-06-10 12:01:33 UTC, end at Fri 2022-06-10 13:55:32 UTC. -- Jun 10 12:56:21.720180 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Jun 10 12:56:21.727692 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 10 12:56:21 2022] *** Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: nodename: dv-cinder-master-3-iscsi Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: machine: x86_64 Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: clock source: unix Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: pcre jit disabled Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: detected number of CPU cores: 4 Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: current working directory: / Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: detected binary path: /usr/bin/uwsgi-core Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: your processes number limit is 47680 Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: your memory page size is 4096 bytes Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: detected max file descriptor number: 2048 Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: lock engine: pthread robust mutexes Jun 10 12:56:21.731177 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: thunder lock: enabled Jun 10 12:56:21.734174 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jun 10 12:56:21.734174 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 10 12:56:21.775021 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: Python main interpreter initialized at 0x55c3910759c0 Jun 10 12:56:21.775021 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: python threads support enabled Jun 10 12:56:21.775021 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: your server socket listen backlog is limited to 100 connections Jun 10 12:56:21.775021 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: your mercy for graceful operations on workers is 90 seconds Jun 10 12:56:21.775373 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: mapped 403077 bytes (393 KB) for 2 cores Jun 10 12:56:21.775450 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: *** Operational MODE: preforking *** Jun 10 12:56:21.775674 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: *** uWSGI is running in multiple interpreter mode *** Jun 10 12:56:21.775674 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: spawned uWSGI master process (pid: 103724) Jun 10 12:56:21.775871 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Jun 10 12:56:21.780907 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: spawned uWSGI worker 1 (pid: 103725, cores: 1) Jun 10 12:56:21.780907 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: spawned uWSGI worker 2 (pid: 103726, cores: 1) Jun 10 12:56:21.780907 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 10 12:56:22.433742 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jun 10 12:56:22.435692 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jun 10 12:56:23.592627 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 10 12:56:23.592627 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: from cryptography.utils import int_from_bytes Jun 10 12:56:23.593263 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 10 12:56:23.593263 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: from cryptography.utils import int_from_bytes Jun 10 12:56:23.595729 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 10 12:56:23.595729 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: from cryptography.utils import int_from_bytes Jun 10 12:56:23.596155 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 10 12:56:23.596155 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: from cryptography.utils import int_from_bytes Jun 10 12:56:26.144114 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 10 12:56:26.144678 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 10 12:56:26.144891 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 10 12:56:26.145241 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 10 12:56:26.145496 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 10 12:56:26.145877 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.146201 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.146303 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 10 12:56:26.146646 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 10 12:56:26.146731 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.146815 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.146963 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 10 12:56:26.147146 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.147389 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 10 12:56:26.147476 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.147649 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 10 12:56:26.147789 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.148038 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.148126 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.148369 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.148453 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.148655 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.148741 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.148987 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.149089 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.149303 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.149398 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.149622 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.149717 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.149924 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.150011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.150230 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.150322 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.150541 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.150636 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.150819 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.150954 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.151132 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.151234 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.151430 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.151529 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.151766 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.151930 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.152072 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.152364 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.152474 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.152667 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.152778 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.152998 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.153085 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.153317 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.153419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.153624 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.153732 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.153943 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.154137 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.154268 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.154448 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.154535 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.154755 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.154858 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.155062 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.155176 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.155397 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.155548 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.155696 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.155850 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.156008 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.156160 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.156327 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.156471 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.156634 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.156902 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.156993 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.157213 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.157301 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.157518 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.157668 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.157834 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.157977 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.158156 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.158282 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.158461 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.158595 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.158782 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.159007 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.159140 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.159319 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.159414 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.159626 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.159713 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.159945 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.160036 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.160277 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.160376 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.160580 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.160674 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.160922 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.161029 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.161227 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.161338 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.161528 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.161634 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.161776 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.161883 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.162079 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.162174 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.162396 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.162511 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.162713 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.163312 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.163558 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.163666 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.163811 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.163977 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.164083 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.164171 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.164362 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.164461 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.164676 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.164795 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.165004 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.165091 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.165313 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.165408 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.165614 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.165702 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.165872 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.165957 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.166122 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.166221 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.166440 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.166536 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.166761 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.166856 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.167069 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.167245 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.167375 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.167460 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.167683 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.167780 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.167995 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.168081 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.168295 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.168387 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.168598 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.168683 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.168931 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.169021 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.169239 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.169335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.169541 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.169625 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.169858 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.169944 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.170178 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.170263 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.170482 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.170566 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.170794 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.170880 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.171114 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.171222 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.171416 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.171513 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.171717 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.171811 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.172019 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.172113 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.172334 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.172430 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.172654 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.172748 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.172976 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.173077 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.173223 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.173319 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.173520 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.173625 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.173820 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.173908 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.174133 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.174229 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.174437 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.174534 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.174736 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.174822 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.175039 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.175126 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.175333 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.175418 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.175633 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.175733 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.175934 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.176020 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.176233 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.176318 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.176533 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.176626 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.176856 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.176960 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.177157 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.177243 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.177455 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.177546 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.177755 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.177849 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.178054 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.178143 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.178357 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.178499 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.179220 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.179308 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.179523 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.179618 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.179825 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.179923 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.180127 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.180214 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.180427 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.180514 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.180730 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.180842 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.181055 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.181166 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.181356 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.181443 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.181671 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.181770 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.181973 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.182066 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.182274 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.182358 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.182578 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.182664 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.182879 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.182968 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.183180 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.183275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.183485 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.183579 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.183779 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.183870 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.184078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.184164 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.184385 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.184497 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.184688 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.184804 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.185008 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.185107 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.185311 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.185410 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.185608 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.185696 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.185905 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.185991 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.186204 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.186298 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.186501 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.186594 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.186800 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.186885 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.187092 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.187180 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.187392 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.187476 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.187696 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.187783 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.187997 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.188085 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.188293 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.188385 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.188592 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.188690 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.188918 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.189007 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.189218 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.189304 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.189535 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.189622 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.189858 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.189965 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.190164 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.190251 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.190485 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.190582 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.190774 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.190861 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.191091 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:56:26.191178 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.191400 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.191496 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.191713 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.191807 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.192032 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.192118 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.192339 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.192433 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.192716 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.192859 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.192977 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.193065 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.193294 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.193386 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.193608 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.193704 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.193933 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.194046 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.194287 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.194376 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.194596 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.194681 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.194906 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.195004 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.195216 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.195310 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.195530 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.195625 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.195848 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.195940 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.196161 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.196248 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.196468 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.196562 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.196802 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.196892 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.197128 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.197217 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.197443 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.197530 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.197761 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.197847 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.198077 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.198164 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.198386 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.198472 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.198692 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.198792 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.198998 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.199086 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.199317 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.199403 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.199649 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.199745 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.199959 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.200054 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.200274 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.200361 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.200582 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.200683 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.200910 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.200998 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.201216 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.201303 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.201529 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.201658 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.201835 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.201934 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.202154 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.202241 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.202463 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.202559 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.202774 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.202857 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.203081 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.203175 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.203386 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.203482 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.203691 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.203784 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.204002 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.204107 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.204308 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.204397 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.204608 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.204697 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.204931 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.205081 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.205238 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.205396 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.205544 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.205714 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.205846 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.206028 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.206161 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.206495 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.206658 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.206771 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.206979 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.207075 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.207294 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.207380 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.207601 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.207689 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.207914 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.208012 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.208226 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.208314 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.208532 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.208620 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.208860 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.208963 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.209169 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.209257 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.209471 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.209557 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.209817 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.209923 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.210128 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.210225 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.210428 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.210527 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.210747 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.210852 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.211051 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.211139 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.211352 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.211440 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.211664 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.211759 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.211973 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.212112 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.212279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.212402 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.212585 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.212710 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.212928 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.213072 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.213204 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.213338 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.213516 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.213642 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.213824 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.213959 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.214161 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.214307 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.214429 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.214556 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.214734 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.214857 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.215050 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.215179 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.215362 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.215453 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.215674 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.215764 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.215976 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.216086 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.216283 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.216380 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.216584 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.216692 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.216964 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.217232 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.217376 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.217513 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.217659 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.217803 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.217949 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.218085 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.218221 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.218370 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.218537 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.218687 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.218851 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.218988 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.219101 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.219183 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.219365 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.219454 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.219671 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.219759 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.219971 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.220064 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.220271 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.220358 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.220571 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.220668 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.220898 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.220989 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.221206 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.221308 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.221512 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.221626 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.221818 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.221917 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.222137 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.222235 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.222446 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.222535 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.222750 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.222845 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.223058 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.223194 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.223360 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.223452 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.223666 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.223753 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.223976 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.224072 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.224288 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.224508 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.224716 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.224963 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.225131 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.225782 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.225941 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.226101 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.226252 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.226391 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.226542 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.226689 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.226829 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.226981 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.227099 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.227184 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.227337 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.227432 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.227650 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.227737 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.227972 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.228058 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.228290 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.228377 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.228611 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.228687 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.228948 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.229030 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.229276 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.229355 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.229587 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.229671 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.229897 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.229981 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.230203 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.230377 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.230519 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.230686 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.230860 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.230995 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.231195 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.231311 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.231513 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.231620 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.231834 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.231985 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.232148 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.232236 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.232463 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.232730 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.232836 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.233064 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.233192 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.233380 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.233466 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.233684 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.233784 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.233997 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.234092 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.234309 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.234438 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.234637 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.234840 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.234995 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.235148 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.235346 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.235482 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.235584 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.235750 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.235836 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.236057 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.236141 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.236370 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.236455 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.236678 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.236755 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.237004 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.237082 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.237310 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.237388 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.237770 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.237870 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.237986 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.238114 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.238303 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.238424 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.238615 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.238732 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.238927 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.239036 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.239233 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.239338 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.239550 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.239704 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.239840 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.239981 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.240140 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.240412 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.240549 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.240720 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.240892 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.241107 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.241236 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.241411 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.241547 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.241730 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.241861 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.242082 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.242250 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.242378 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.242501 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.242621 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.242740 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.242931 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.243071 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.243235 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.243363 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.243535 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.243671 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.243836 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.243973 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.244135 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.244262 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.244447 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.244570 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.244830 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.244979 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.245114 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.245234 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.245387 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.245523 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.245693 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.245821 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.245996 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.246141 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.246306 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.246430 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.246656 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.246781 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.246912 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.247033 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.247214 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.247339 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.247524 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.247662 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.247827 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.247955 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.248134 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.248260 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.248441 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.248578 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.248747 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.248899 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.249077 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.249220 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.249392 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.249518 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.249695 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.249830 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.250002 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.250150 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.250308 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.250434 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.250610 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.250738 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.250916 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.251057 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.251231 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.251359 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.251537 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.251812 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.251942 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.252142 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.252282 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.252444 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.252569 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.252753 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.252947 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.253082 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.253210 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.253392 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.253647 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.253777 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.253983 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.254113 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.254300 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.254428 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.254618 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.254770 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.254928 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.255064 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.255227 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.255353 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.255533 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.255657 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.255825 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.255962 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.256127 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.256256 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.256427 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.257123 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.257260 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.257382 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.257569 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.257709 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.257867 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.257994 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.258165 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.258292 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.258463 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.258590 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.258764 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.258890 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.259071 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.259197 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.259371 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.259510 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.259678 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.259814 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.259974 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.260111 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.260274 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.260399 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.260582 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.260706 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.260902 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.261046 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.261218 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.261356 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.261503 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.261889 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.262057 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.262194 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.262382 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.262513 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.262692 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.262823 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.262994 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.263132 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.263297 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.263422 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.263602 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.263727 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.263928 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.264054 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.264232 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.264357 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.264538 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.264710 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.264889 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.265014 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.265134 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.265255 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.265439 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.265619 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.265749 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.265871 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.266033 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.266161 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.266341 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.266470 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.266647 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.266775 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.266952 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.267080 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.267264 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.267404 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.267563 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.267693 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.267869 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.268008 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.268168 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.268312 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.268477 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.268605 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.268839 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.268969 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.269119 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.269265 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.269425 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.269558 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.269737 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.269850 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.270041 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.270162 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.270337 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.270422 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.270645 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.270740 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.270942 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.271060 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.271246 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.271371 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.271543 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.271666 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.271841 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.271977 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.272179 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.272829 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.273034 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.273189 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.273330 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.273461 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.273626 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.273712 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.273936 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.274037 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.274243 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.274371 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.274543 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.274670 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.274851 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.274991 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.275138 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.275262 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.275438 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.275571 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.275741 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.275887 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.276025 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.276153 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.276345 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.276435 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.276650 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.276754 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.276973 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.277118 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.277278 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.277402 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.277575 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.277717 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.277859 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.277964 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.278178 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.278311 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.278453 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.278582 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.278751 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.278884 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.279046 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.279134 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.279346 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.279443 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.279639 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.279727 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.279942 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.280092 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.280256 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.280390 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.280558 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.280680 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.280881 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.281022 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.281169 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.281299 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.281467 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.281600 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.281768 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.281901 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.282094 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.282214 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.282406 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.282500 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.282713 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.282809 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.283021 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.283123 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.283325 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.283412 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.283631 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.283723 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.283940 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.284027 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.284251 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.284344 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.284552 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.284647 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.284885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.284987 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.285193 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.285280 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.285503 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.285592 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.285801 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.285895 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.286101 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.286195 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.286421 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.286508 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.286727 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.286813 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.287037 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.287121 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.287351 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.287445 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.287657 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.287743 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.288006 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.288092 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.288306 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.288393 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.288611 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.288697 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.288941 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.289030 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.289248 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.289336 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.289554 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.289640 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.289859 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.289998 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.290167 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.290262 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.290474 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.290560 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.290784 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.290878 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.291093 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.291186 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.291402 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.291494 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.291704 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.291797 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.292009 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.292095 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.292314 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.292401 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.292615 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.292711 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.292953 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.293087 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.293260 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.293345 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.293559 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.293647 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.293865 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.293959 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.294165 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.294252 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.294471 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.294557 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.294766 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.294853 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.295097 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.295191 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.295403 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.295491 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.295712 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.295797 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.296017 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.296117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.296329 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.296416 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.296635 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.296720 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.296989 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.297088 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.297303 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.297389 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.297604 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.297689 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.297986 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.298079 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.298300 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.298387 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.298605 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.298690 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.298912 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.299008 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.299214 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.299301 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.299514 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.299609 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.299820 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.299984 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.300140 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.300229 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.300441 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.300536 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.300740 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.300847 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.301059 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.301145 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.301378 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.301466 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.301678 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.301767 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.301995 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.302081 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.302340 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.302428 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.302647 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.302734 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.302952 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.303038 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.303276 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.303391 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.303608 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.303696 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.303934 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.304027 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.304308 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.304401 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.304630 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.304716 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.304968 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.305091 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.305269 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.305387 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.305580 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.305700 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.305886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.306011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.306206 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.306338 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.306526 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.306663 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.306834 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.306988 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.307149 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.307297 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.307450 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.307613 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.307770 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.307924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.308089 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.308250 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.308400 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.308578 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.308719 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.308920 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.309152 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.309240 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.309473 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.309563 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.309781 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.309867 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.310104 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.310228 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.310426 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.310530 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.310731 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.310826 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.311033 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.311164 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.311341 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.311598 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.311685 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.311918 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.312005 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.312239 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.312326 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.312543 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.312630 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.312878 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.312970 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.313183 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.313316 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.313488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.313574 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.313789 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.313885 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.314089 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.314178 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.314409 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.314496 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.314711 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.314815 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.315022 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.315109 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.315333 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.315428 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.315641 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.315726 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.315946 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.316035 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.316244 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.316330 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.316563 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.316664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.316893 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.316983 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.317193 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.317278 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.317495 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.317600 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.317797 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.317891 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.318135 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.318223 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.318434 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.318531 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.318753 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.318832 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.320806 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.320938 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.321017 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.321101 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.321172 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.321239 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.321307 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.321373 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.321449 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.321519 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.321591 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.321659 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.321725 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.321799 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.321880 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.321955 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.322064 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.322175 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.322380 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.322717 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.322908 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.323042 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.323233 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.323530 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.323627 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.323834 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.323936 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.324134 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.324242 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.324467 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.324680 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.324906 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.325002 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.325085 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.325302 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.325400 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.325607 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.325706 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.325929 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.326025 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.326233 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.326320 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.326554 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.326652 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.326857 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.326943 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.327159 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.327244 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.327467 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.327557 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.327771 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.327857 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.328095 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.328182 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.328401 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.328488 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.328706 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.328812 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.329043 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.329129 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.329356 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.329445 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.329679 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.329766 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.329989 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.330126 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.332546 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.332702 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.332888 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.333005 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.333110 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.333224 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.333329 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.333432 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.333533 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.333635 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.333737 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.333854 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.333969 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.334073 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.334176 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.334279 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.335501 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.335590 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.335670 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.335749 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.335827 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.335904 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.335991 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.336072 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.336072 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.336230 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.336317 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.336396 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.336482 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.336568 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.336654 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.336733 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.336831 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.336930 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.336930 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.337085 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.337164 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.337241 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.337319 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.337405 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.337486 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.337563 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.337641 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.337718 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.337796 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.337882 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.338109 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.338198 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.338429 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.338525 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.338754 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.338842 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.339091 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.339186 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.339405 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.339492 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.339720 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.339809 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.340034 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.340121 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.340347 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.340433 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.340685 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.340806 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.341031 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.341120 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.341336 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.341440 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.341649 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.341737 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.341961 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.342049 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.342273 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.342368 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.342579 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.342672 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.342886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.342981 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.343198 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.343286 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.343507 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.343601 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.343815 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.343902 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.344121 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.344209 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.344471 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.344571 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.344733 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.344840 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.345071 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.345159 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.345383 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.345516 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.345695 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.345792 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.346002 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.346089 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.346314 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.346403 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.346626 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.346726 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.346958 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.347146 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.347277 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.347463 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.347591 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.347793 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.347896 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.348116 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.348204 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.348438 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.348712 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.348820 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.349063 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.349151 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.349389 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.349474 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.349693 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.349781 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.349996 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.350083 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.350340 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.350431 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.350654 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.350742 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.350956 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.351043 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.351276 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.351361 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.351584 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.351673 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.351885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.351972 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.352194 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.352288 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.352498 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.352583 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.352839 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.353053 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.353143 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.353397 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.353486 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.353709 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.353796 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.354031 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.354118 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.354350 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.354576 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.354675 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.354912 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.355007 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.355232 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.355321 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.355545 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.355630 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.355847 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.355934 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.356157 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.356246 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.356465 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.356553 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.356790 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.356880 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.357089 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.357178 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.357394 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.357490 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.357700 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.357787 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.357998 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.358087 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.358302 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.358395 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.358599 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.358698 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.358910 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.358996 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.359210 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.359296 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.359510 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.359609 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.359809 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.359896 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.360109 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.360194 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.360407 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.360503 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.360708 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.360851 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.361034 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.361130 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.361333 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.361429 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.361640 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.361735 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.361939 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.362025 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.362240 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.362335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.362539 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.362624 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.362841 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.362926 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.363139 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.363226 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.363441 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.363526 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.363750 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.363836 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.364055 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.364143 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.364365 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.364452 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.364680 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.364818 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.365029 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.365118 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.365364 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.365450 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.365683 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.365777 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.366515 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.366609 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.366823 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.366911 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.367137 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.367233 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.367453 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.367539 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.367779 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.367995 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.368085 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.368328 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.368417 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.368644 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.368733 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.368998 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.369095 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.369328 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.369577 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.369663 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.369913 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.370007 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.370215 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.370378 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.370536 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.370624 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.370842 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.370928 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.371148 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.371236 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.371458 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.371546 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.371761 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.371846 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.372065 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.372153 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.372371 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.372458 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.372678 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.372812 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.373006 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.373104 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.373311 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.373460 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.373648 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.373781 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.373921 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.374009 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.374225 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.374312 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.374528 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.374623 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.374837 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.374934 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.375145 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.375232 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.375454 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.375555 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.375764 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.375851 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.376071 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.376157 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.376382 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.376477 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.376690 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.376805 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.377031 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.377118 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.377337 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.377427 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.377642 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.377730 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.377953 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.378086 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.378269 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.378372 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.378573 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.378668 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.378876 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.378975 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.379188 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.379275 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.379505 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.379591 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.379834 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.379935 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.380150 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.380236 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.380456 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.380543 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.380803 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.380949 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.381125 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.381213 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.381432 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.381586 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.381799 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.381926 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.382107 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.382239 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.382432 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.382562 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.382766 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.382888 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.383091 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.383190 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.383410 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.383504 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.383712 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.383800 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.384012 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.384111 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.384313 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.384447 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.384618 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.384850 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.384962 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.385125 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.385248 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.385434 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.385547 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.385740 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.385828 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.386046 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.386135 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.386355 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.386444 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.386929 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.387029 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.387241 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.387328 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.387538 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.387624 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.387838 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.387923 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.388133 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.388228 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.388429 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.388516 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.388726 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.388834 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.389045 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.389148 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.389342 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.389430 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.389638 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.389724 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.389939 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.390033 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.390234 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.390319 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.390532 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.390629 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.390828 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.390924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.391122 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.391220 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.391417 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.391506 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.391722 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.391817 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.392024 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.392111 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.392328 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.392422 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.392629 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.392718 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.392967 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.393055 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.393268 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.393354 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.393575 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.393663 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.393892 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.393987 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.394254 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.394356 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.394519 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.394615 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.394823 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.394933 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.395145 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.395253 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.395447 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.395540 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.395755 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.395843 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.396063 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.396159 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.396371 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.396471 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.396675 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.396812 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.397009 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.397111 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.397865 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.397960 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.398178 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.398265 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.398479 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.398578 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.398791 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.398878 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.399102 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.399188 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.399413 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.399503 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.399724 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.399809 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.400032 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.400120 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.400350 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.400438 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.400656 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.400741 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.400987 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.401079 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.401301 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.401405 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.401605 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.401694 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.401913 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.402012 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.402227 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.402314 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.402540 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.402626 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.402852 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.402947 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.403169 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.403259 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.403473 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.403557 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.403783 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.403878 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.404102 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.404198 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.404418 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.404513 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.404739 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.404881 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.405088 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.405243 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.405400 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.405487 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.405710 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.405806 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.406019 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.406151 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.406322 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.406410 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.406563 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103725) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 10 12:56:26.406673 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.407017 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.407349 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.407667 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.407981 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.408289 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:56:26.408533 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103726) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 10 12:56:26.468896 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO dbcounter [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] Registered counter for database nova_api Jun 10 12:56:26.470491 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO dbcounter [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] Registered counter for database nova_api Jun 10 12:56:26.489566 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG oslo_db.sqlalchemy.engines [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103725) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 12:56:26.493468 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_db.sqlalchemy.engines [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103726) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 12:56:26.496669 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG dbcounter [-] [103725] Writer thread running {{(pid=103725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 12:56:26.499875 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG dbcounter [-] [103726] Writer thread running {{(pid=103726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 12:56:26.557112 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.context [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),bf155fd7-c737-40a3-a522-fd4049b0e8da(cell1) {{(pid=103725) load_cells /opt/stack/nova/nova/context.py:464}} Jun 10 12:56:26.558669 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.context [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),bf155fd7-c737-40a3-a522-fd4049b0e8da(cell1) {{(pid=103726) load_cells /opt/stack/nova/nova/context.py:464}} Jun 10 12:56:26.562533 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG oslo_concurrency.lockutils [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103725) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 12:56:26.562789 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 12:56:26.564350 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 12:56:26.564441 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG oslo_concurrency.lockutils [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103725) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 12:56:26.565174 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 12:56:26.565861 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG oslo_concurrency.lockutils [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103725) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 12:56:26.567138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 12:56:26.568204 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG oslo_concurrency.lockutils [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103725) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 12:56:26.569781 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO dbcounter [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] Registered counter for database nova_cell0 Jun 10 12:56:26.571259 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO dbcounter [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] Registered counter for database nova_cell0 Jun 10 12:56:26.573745 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO dbcounter [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] Registered counter for database nova_cell1 Jun 10 12:56:26.575996 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO dbcounter [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] Registered counter for database nova_cell1 Jun 10 12:56:26.581333 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_db.sqlalchemy.engines [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103726) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 12:56:26.584439 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_db.sqlalchemy.engines [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103726) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 12:56:26.585061 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG oslo_db.sqlalchemy.engines [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103725) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 12:56:26.587735 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG dbcounter [-] [103726] Writer thread running {{(pid=103726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 12:56:26.588265 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG oslo_db.sqlalchemy.engines [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103725) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 12:56:26.591239 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG dbcounter [-] [103725] Writer thread running {{(pid=103725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 12:56:26.725002 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG dbcounter [-] [103726] Writer thread running {{(pid=103726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 12:56:26.730921 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG dbcounter [-] [103725] Writer thread running {{(pid=103725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 12:56:26.735249 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO dbcounter [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] Registered counter for database nova_cell0 Jun 10 12:56:26.739974 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO dbcounter [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] Registered counter for database nova_cell0 Jun 10 12:56:26.744109 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_db.sqlalchemy.engines [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103726) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 12:56:26.746523 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG oslo_db.sqlalchemy.engines [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103725) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 12:56:26.748610 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG dbcounter [-] [103726] Writer thread running {{(pid=103726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 12:56:26.750950 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG dbcounter [-] [103725] Writer thread running {{(pid=103725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 12:56:27.203591 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: WARNING keystonemiddleware.auth_token [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jun 10 12:56:27.211233 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: WARNING keystonemiddleware.auth_token [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jun 10 12:56:27.219993 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: WARNING keystonemiddleware.auth_token [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jun 10 12:56:27.227144 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: WARNING keystonemiddleware.auth_token [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jun 10 12:56:27.554305 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: WARNING keystonemiddleware.auth_token [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jun 10 12:56:27.560091 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: WARNING keystonemiddleware.auth_token [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jun 10 12:56:27.565356 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c3910759c0 pid: 103726 (default app) Jun 10 12:56:27.567402 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:56:27.568573 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] 10.222.0.25 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001708 Jun 10 12:56:27.569038 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 788 bytes} [Fri Jun 10 12:56:27 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:56:27.580055 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: WARNING keystonemiddleware.auth_token [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jun 10 12:56:27.586129 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: WARNING keystonemiddleware.auth_token [None req-ea10b42a-062e-45d8-9f62-08edfdb07614 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jun 10 12:56:27.591350 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c3910759c0 pid: 103725 (default app) Jun 10 12:57:46.916177 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:57:46.917405 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001677 Jun 10 12:57:46.917796 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 2/2] 10.222.0.25 () {54 vars in 930 bytes} [Fri Jun 10 12:57:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:57:46.929930 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG dbcounter [-] [103725] Writing DB stats nova_api:SELECT=3 {{(pid=103725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:57:46.931029 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG dbcounter [-] [103725] Writing DB stats nova_cell0:SELECT=3 {{(pid=103725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:57:46.931874 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG dbcounter [-] [103725] Writing DB stats nova_cell1:SELECT=3 {{(pid=103725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:57:46.932932 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG dbcounter [-] [103725] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:57:47.370741 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-b4487ea2-02c0-4991-9e69-9b5482171514 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:57:47.401638 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-b4487ea2-02c0-4991-9e69-9b5482171514 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.477528 Jun 10 12:57:47.402416 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 1/3] 10.222.0.25 () {60 vars in 1291 bytes} [Fri Jun 10 12:57:46 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 12:57:49.565934 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:57:49.566951 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-9bd1c3fc-1ea9-42ad-a030-b112cb538a86 None None] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001453 Jun 10 12:57:49.567366 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 3/4] 10.222.0.25 () {54 vars in 930 bytes} [Fri Jun 10 12:57:49 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:57:49.633008 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-dc635541-eda8-4a34-ac21-7b970e4f0b9e admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "cirros256", "rxtx_factor": 1.0, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "c1", "os-flavor-access:is_public": true, "ram": 256, "disk": 1, "swap": 0}} {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:57:49.651945 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 10 12:57:49.651945 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: warnings.warn(msg) Jun 10 12:57:49.787252 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-dc635541-eda8-4a34-ac21-7b970e4f0b9e admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.208430 Jun 10 12:57:49.787252 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 2/5] 10.222.0.25 () {64 vars in 1282 bytes} [Fri Jun 10 12:57:49 2022] POST /compute/v2.1/flavors => generated 407 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 12:57:49.797183 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG dbcounter [-] [103726] Writing DB stats nova_api:SELECT=3 {{(pid=103726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:57:49.797997 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG dbcounter [-] [103726] Writing DB stats nova_cell0:SELECT=3 {{(pid=103726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:57:49.800620 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG dbcounter [-] [103726] Writing DB stats nova_cell1:SELECT=3 {{(pid=103726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:57:49.801571 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG dbcounter [-] [103726] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:57:50.016294 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-aadb3af6-8047-43d4-b9c4-4f40efd14fa1 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:57:50.021101 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 10 12:57:50.021101 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: warnings.warn(msg) Jun 10 12:57:50.441825 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-aadb3af6-8047-43d4-b9c4-4f40efd14fa1 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.649709 Jun 10 12:57:50.442594 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 4/6] 10.222.0.25 () {64 vars in 1335 bytes} [Fri Jun 10 12:57:49 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 651 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 12:57:52.589498 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-dc635541-eda8-4a34-ac21-7b970e4f0b9e admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:57:52.590467 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-dc635541-eda8-4a34-ac21-7b970e4f0b9e admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001508 Jun 10 12:57:52.590877 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 3/7] 10.222.0.25 () {54 vars in 930 bytes} [Fri Jun 10 12:57:52 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:57:52.695249 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-c0fe4d5f-1ab6-4f27-ba9b-24cd25c5f2a8 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "id": "d1", "swap": 0, "disk": 5, "ram": 512, "name": "ds512M", "os-flavor-access:is_public": true, "rxtx_factor": 1.0}} {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:57:52.712826 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 10 12:57:52.712826 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: warnings.warn(msg) Jun 10 12:57:52.799897 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-c0fe4d5f-1ab6-4f27-ba9b-24cd25c5f2a8 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.202384 Jun 10 12:57:52.800465 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 5/8] 10.222.0.25 () {64 vars in 1282 bytes} [Fri Jun 10 12:57:52 2022] POST /compute/v2.1/flavors => generated 404 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 12:57:52.812291 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-8556ea49-5c7c-41b6-8370-796e5f502be9 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:57:52.813897 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 10 12:57:52.813897 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: warnings.warn(msg) Jun 10 12:57:52.947464 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-8556ea49-5c7c-41b6-8370-796e5f502be9 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.138853 Jun 10 12:57:52.948585 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 4/9] 10.222.0.25 () {64 vars in 1335 bytes} [Fri Jun 10 12:57:52 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 12:57:55.113587 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-c0fe4d5f-1ab6-4f27-ba9b-24cd25c5f2a8 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:57:55.114618 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-c0fe4d5f-1ab6-4f27-ba9b-24cd25c5f2a8 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001536 Jun 10 12:57:55.115046 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 6/10] 10.222.0.25 () {54 vars in 930 bytes} [Fri Jun 10 12:57:55 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 12:57:55.177588 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-9e394ed9-a810-482a-9516-8c0ec1abc42a admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "name": "ds1G", "disk": 10, "os-flavor-access:is_public": true, "ram": 1024, "rxtx_factor": 1.0, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d2"}} {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:57:55.320045 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-9e394ed9-a810-482a-9516-8c0ec1abc42a admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.198181 Jun 10 12:57:55.321304 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 5/11] 10.222.0.25 () {64 vars in 1282 bytes} [Fri Jun 10 12:57:55 2022] POST /compute/v2.1/flavors => generated 404 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 12:57:55.336827 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-3de2f821-0aa7-4bda-95a0-7a4fd54da98e admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:57:55.419796 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-3de2f821-0aa7-4bda-95a0-7a4fd54da98e admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.089864 Jun 10 12:57:55.420540 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 7/12] 10.222.0.25 () {64 vars in 1335 bytes} [Fri Jun 10 12:57:55 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 12:57:57.618785 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-9e394ed9-a810-482a-9516-8c0ec1abc42a admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:57:57.619839 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-9e394ed9-a810-482a-9516-8c0ec1abc42a admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001499 Jun 10 12:57:57.620245 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 6/13] 10.222.0.25 () {54 vars in 930 bytes} [Fri Jun 10 12:57:57 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 12:57:57.680437 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-0f2b4e97-dc12-4f3a-a6cb-22f8d33a27ce admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "vcpus": 2, "disk": 10, "ram": 2048, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "id": "d3", "name": "ds2G"}} {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:57:57.776702 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-0f2b4e97-dc12-4f3a-a6cb-22f8d33a27ce admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.150335 Jun 10 12:57:57.777323 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 8/14] 10.222.0.25 () {64 vars in 1282 bytes} [Fri Jun 10 12:57:57 2022] POST /compute/v2.1/flavors => generated 404 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 12:57:57.789278 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-57e17460-17f9-44b1-a3df-a261f5bcc41a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:57:57.870219 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-57e17460-17f9-44b1-a3df-a261f5bcc41a admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.084686 Jun 10 12:57:57.870861 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 7/15] 10.222.0.25 () {64 vars in 1335 bytes} [Fri Jun 10 12:57:57 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 12:58:00.055414 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-0f2b4e97-dc12-4f3a-a6cb-22f8d33a27ce admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:00.056430 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-0f2b4e97-dc12-4f3a-a6cb-22f8d33a27ce admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001450 Jun 10 12:58:00.057038 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 9/16] 10.222.0.25 () {54 vars in 930 bytes} [Fri Jun 10 12:58:00 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:00.116504 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-ebdc0582-6238-4fea-be27-0637f1aea4c1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "name": "ds4G", "ram": 4096, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 20, "vcpus": 4, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "id": "d4"}} {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:00.213326 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-ebdc0582-6238-4fea-be27-0637f1aea4c1 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.150820 Jun 10 12:58:00.213888 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 8/17] 10.222.0.25 () {64 vars in 1282 bytes} [Fri Jun 10 12:58:00 2022] POST /compute/v2.1/flavors => generated 404 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 12:58:00.225860 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-02b86f44-27d4-4eca-b82e-90547dc3c2ae admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:00.307423 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-02b86f44-27d4-4eca-b82e-90547dc3c2ae admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.085180 Jun 10 12:58:00.308082 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 10/18] 10.222.0.25 () {64 vars in 1335 bytes} [Fri Jun 10 12:58:00 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 12:58:02.474323 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-ebdc0582-6238-4fea-be27-0637f1aea4c1 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:02.475347 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-ebdc0582-6238-4fea-be27-0637f1aea4c1 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001479 Jun 10 12:58:02.475757 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 9/19] 10.222.0.25 () {54 vars in 930 bytes} [Fri Jun 10 12:58:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:02.538173 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-a047a4ba-9183-4ee0-9ece-c50f06c1057a admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:02.562460 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-a047a4ba-9183-4ee0-9ece-c50f06c1057a admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.080731 Jun 10 12:58:02.568814 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 11/20] 10.222.0.25 () {60 vars in 1291 bytes} [Fri Jun 10 12:58:02 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 12:58:05.139969 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-ebdc0582-6238-4fea-be27-0637f1aea4c1 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:05.141068 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-ebdc0582-6238-4fea-be27-0637f1aea4c1 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001516 Jun 10 12:58:05.141489 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 10/21] 10.222.0.25 () {54 vars in 930 bytes} [Fri Jun 10 12:58:05 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:05.203378 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-716d25fa-09b6-4571-b5e2-2882dd611550 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "ram": 512, "rxtx_factor": 1.0, "name": "m1.tiny", "vcpus": 1, "swap": 0, "id": "1"}} {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:05.280960 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-716d25fa-09b6-4571-b5e2-2882dd611550 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.132903 Jun 10 12:58:05.281535 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 12/22] 10.222.0.25 () {64 vars in 1282 bytes} [Fri Jun 10 12:58:05 2022] POST /compute/v2.1/flavors => generated 402 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 12:58:05.297177 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-dc69770d-798f-40c3-b69f-70b77b3d7340 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:05.378478 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-dc69770d-798f-40c3-b69f-70b77b3d7340 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.085176 Jun 10 12:58:05.379302 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 11/23] 10.222.0.25 () {64 vars in 1332 bytes} [Fri Jun 10 12:58:05 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 12:58:07.679353 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-716d25fa-09b6-4571-b5e2-2882dd611550 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:07.680372 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-716d25fa-09b6-4571-b5e2-2882dd611550 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001468 Jun 10 12:58:07.680840 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 13/24] 10.222.0.25 () {54 vars in 930 bytes} [Fri Jun 10 12:58:07 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:07.743229 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-d3f13fdf-3897-401a-b2a0-b7dd191b4b7f admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "swap": 0, "id": "2", "name": "m1.small", "vcpus": 1, "rxtx_factor": 1.0, "ram": 2048, "disk": 20, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:07.832440 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-d3f13fdf-3897-401a-b2a0-b7dd191b4b7f admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.145506 Jun 10 12:58:07.833169 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 12/25] 10.222.0.25 () {64 vars in 1282 bytes} [Fri Jun 10 12:58:07 2022] POST /compute/v2.1/flavors => generated 405 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 12:58:07.845305 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-b65fa99b-4538-4a68-a6be-ecdd3199a039 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:07.933009 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-b65fa99b-4538-4a68-a6be-ecdd3199a039 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.091229 Jun 10 12:58:07.933524 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 14/26] 10.222.0.25 () {64 vars in 1332 bytes} [Fri Jun 10 12:58:07 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 12:58:10.144201 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-d3f13fdf-3897-401a-b2a0-b7dd191b4b7f admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:10.145266 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-d3f13fdf-3897-401a-b2a0-b7dd191b4b7f admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001492 Jun 10 12:58:10.145678 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 13/27] 10.222.0.25 () {54 vars in 930 bytes} [Fri Jun 10 12:58:10 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:10.204342 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-bc29975c-00da-4f0b-b28d-19d3094100d4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "ram": 4096, "disk": 40, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 2, "name": "m1.medium", "id": "3"}} {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:10.272424 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-bc29975c-00da-4f0b-b28d-19d3094100d4 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.121127 Jun 10 12:58:10.273036 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 15/28] 10.222.0.25 () {64 vars in 1282 bytes} [Fri Jun 10 12:58:10 2022] POST /compute/v2.1/flavors => generated 406 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 12:58:10.285337 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-057c5e98-ad37-44ce-a46e-5ffb163b19d3 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:10.361609 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-057c5e98-ad37-44ce-a46e-5ffb163b19d3 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.080481 Jun 10 12:58:10.362195 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 14/29] 10.222.0.25 () {64 vars in 1332 bytes} [Fri Jun 10 12:58:10 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 12:58:12.515217 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-bc29975c-00da-4f0b-b28d-19d3094100d4 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:12.516222 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-bc29975c-00da-4f0b-b28d-19d3094100d4 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001465 Jun 10 12:58:12.516662 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 16/30] 10.222.0.25 () {54 vars in 930 bytes} [Fri Jun 10 12:58:12 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:12.572582 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-d554f9c3-7e82-4a54-946d-8406f9467284 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.large", "rxtx_factor": 1.0, "swap": 0, "disk": 80, "ram": 8192, "id": "4", "vcpus": 4}} {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:12.638704 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-d554f9c3-7e82-4a54-946d-8406f9467284 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.115920 Jun 10 12:58:12.639197 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 15/31] 10.222.0.25 () {64 vars in 1282 bytes} [Fri Jun 10 12:58:12 2022] POST /compute/v2.1/flavors => generated 405 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 12:58:12.654245 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-f647b162-9f6d-4d2a-ac08-354b143c224e admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:12.738714 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-f647b162-9f6d-4d2a-ac08-354b143c224e admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.089210 Jun 10 12:58:12.739309 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 17/32] 10.222.0.25 () {64 vars in 1332 bytes} [Fri Jun 10 12:58:12 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 12:58:14.926735 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-d554f9c3-7e82-4a54-946d-8406f9467284 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:14.927751 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-d554f9c3-7e82-4a54-946d-8406f9467284 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001455 Jun 10 12:58:14.928148 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 16/33] 10.222.0.25 () {54 vars in 930 bytes} [Fri Jun 10 12:58:14 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:14.987354 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-564426db-72f3-4e0a-81f0-6c68d8f37861 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 16384, "id": "5", "swap": 0, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 160, "name": "m1.xlarge", "rxtx_factor": 1.0, "vcpus": 8}} {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:15.072616 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-564426db-72f3-4e0a-81f0-6c68d8f37861 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.138729 Jun 10 12:58:15.073221 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 18/34] 10.222.0.25 () {64 vars in 1282 bytes} [Fri Jun 10 12:58:14 2022] POST /compute/v2.1/flavors => generated 408 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 12:58:15.085514 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-b64e2dc0-965d-4b61-a9fe-8fd5c1859b29 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:15.198270 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-b64e2dc0-965d-4b61-a9fe-8fd5c1859b29 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.116383 Jun 10 12:58:15.226991 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 17/35] 10.222.0.25 () {64 vars in 1332 bytes} [Fri Jun 10 12:58:15 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 12:58:39.087285 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-564426db-72f3-4e0a-81f0-6c68d8f37861 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:39.088299 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-564426db-72f3-4e0a-81f0-6c68d8f37861 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001446 Jun 10 12:58:39.088693 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 19/36] 10.222.0.25 () {54 vars in 930 bytes} [Fri Jun 10 12:58:39 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:39.099088 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG dbcounter [-] [103725] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=10,nova_api:SAVEPOINT=5,nova_api:RELEASE=5 {{(pid=103725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:58:39.146278 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-9356ef41-e88d-4243-b92f-7487f14e03e8 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:39.163337 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-9356ef41-e88d-4243-b92f-7487f14e03e8 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.068847 Jun 10 12:58:39.163868 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 18/37] 10.222.0.25 () {60 vars in 1291 bytes} [Fri Jun 10 12:58:39 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 12:58:40.121900 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-564426db-72f3-4e0a-81f0-6c68d8f37861 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:40.122906 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-564426db-72f3-4e0a-81f0-6c68d8f37861 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001445 Jun 10 12:58:40.123308 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 20/38] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Jun 10 12:58:40 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:40.327361 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-e867675e-e44a-41ba-aed2-5c3ec39a6dc0 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:40.333722 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.wsgi [None req-e867675e-e44a-41ba-aed2-5c3ec39a6dc0 admin admin] HTTP exception thrown: Flavor 42 could not be found. Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-e867675e-e44a-41ba-aed2-5c3ec39a6dc0 admin admin] Request method failure captured: Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Accept: application/json Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Accept-Encoding: gzip, deflate Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Connection: keep-alive Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Content-Length: 0 Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Host: 10.222.0.25 Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Openstack-Api-Version: compute 2.90 Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Openstack-System-Scope: None Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: User-Agent: python-novaclient Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Auth-Token: gAAAAABio0AAkLZL1cD9DH9B1kGByM0YRL8NbQ9yVbYAPaqZrWtzgK8Z1EO0YfRwJMsvkfS60lQeIlcIeAEr5B31nOB2R_BAJzxO2AwM1PeCapF2q9sCwcQJd3eyWxv12MFS7xh5WLC9QfqBSve-JtZV0ULLF0RGuE0ODhEqC7LXoho-3J8gO-0 Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Domain-Id: None Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Domain-Name: None Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Identity-Status: Confirmed Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Is-Admin-Project: True Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Openstack-Nova-Api-Version: 2.90 Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Project-Domain-Id: default Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Project-Domain-Name: Default Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Project-Id: 96340edb86a840a88d51d74872a27bac Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Project-Name: admin Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Role: reader,member,admin Jun 10 12:58:40.336886 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Roles: reader,member,admin Jun 10 12:58:40.338595 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/96340edb86a840a88d51d74872a27bac"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/96340edb86a840a88d51d74872a27bac"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_96340edb86a840a88d51d74872a27bac", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/96340edb86a840a88d51d74872a27bac"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 12:58:40.338595 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Tenant: admin Jun 10 12:58:40.338595 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Tenant-Id: 96340edb86a840a88d51d74872a27bac Jun 10 12:58:40.338595 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Tenant-Name: admin Jun 10 12:58:40.338595 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User: admin Jun 10 12:58:40.338595 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User-Domain-Id: default Jun 10 12:58:40.338595 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User-Domain-Name: Default Jun 10 12:58:40.338595 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User-Id: 05ecde7b5ffd440ea8bdbb9c0c6418c1 Jun 10 12:58:40.338595 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User-Name: admin Jun 10 12:58:40.338595 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: method: > Jun 10 12:58:40.338595 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: exception: Flavor 42 could not be found. Jun 10 12:58:40.338595 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: action_args: {'id': '42'} Jun 10 12:58:40.338595 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 12:58:40.338595 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 12:58:40.339567 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:40.341117 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-e867675e-e44a-41ba-aed2-5c3ec39a6dc0 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103725) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 12:58:40.342381 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-e867675e-e44a-41ba-aed2-5c3ec39a6dc0 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.078555 Jun 10 12:58:40.342381 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 19/39] 10.222.0.25 () {60 vars in 1190 bytes} [Fri Jun 10 12:58:40 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 80 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jun 10 12:58:40.348530 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-750865e9-20f4-4803-a5de-b4b4ead4d64d admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:40.351600 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG dbcounter [-] [103726] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=103726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:58:40.370177 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-750865e9-20f4-4803-a5de-b4b4ead4d64d admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.025282 Jun 10 12:58:40.370740 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 21/40] 10.222.0.25 () {60 vars in 1210 bytes} [Fri Jun 10 12:58:40 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 12:58:40.380358 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-4d50fdd7-578d-41ff-aaff-cd01c2798429 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:40.402226 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-4d50fdd7-578d-41ff-aaff-cd01c2798429 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.025552 Jun 10 12:58:40.402771 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 20/41] 10.222.0.25 () {60 vars in 1210 bytes} [Fri Jun 10 12:58:40 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 12:58:40.412743 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-811c2117-a73c-490c-a794-f50b98887bcc admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:40.418602 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.wsgi [None req-811c2117-a73c-490c-a794-f50b98887bcc admin admin] HTTP exception thrown: Flavor 42 could not be found. Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-811c2117-a73c-490c-a794-f50b98887bcc admin admin] Request method failure captured: Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Accept: application/json Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Accept-Encoding: gzip, deflate Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Connection: keep-alive Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Content-Length: 0 Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Host: 10.222.0.25 Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Openstack-Api-Version: compute 2.90 Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Openstack-System-Scope: None Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: User-Agent: python-novaclient Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Auth-Token: gAAAAABio0AAkLZL1cD9DH9B1kGByM0YRL8NbQ9yVbYAPaqZrWtzgK8Z1EO0YfRwJMsvkfS60lQeIlcIeAEr5B31nOB2R_BAJzxO2AwM1PeCapF2q9sCwcQJd3eyWxv12MFS7xh5WLC9QfqBSve-JtZV0ULLF0RGuE0ODhEqC7LXoho-3J8gO-0 Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Domain-Id: None Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Domain-Name: None Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Identity-Status: Confirmed Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Is-Admin-Project: True Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Openstack-Nova-Api-Version: 2.90 Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Project-Domain-Id: default Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Project-Domain-Name: Default Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Project-Id: 96340edb86a840a88d51d74872a27bac Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Project-Name: admin Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Role: reader,member,admin Jun 10 12:58:40.421924 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Roles: reader,member,admin Jun 10 12:58:40.427011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/96340edb86a840a88d51d74872a27bac"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/96340edb86a840a88d51d74872a27bac"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_96340edb86a840a88d51d74872a27bac", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/96340edb86a840a88d51d74872a27bac"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 12:58:40.427011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Tenant: admin Jun 10 12:58:40.427011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Tenant-Id: 96340edb86a840a88d51d74872a27bac Jun 10 12:58:40.427011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Tenant-Name: admin Jun 10 12:58:40.427011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User: admin Jun 10 12:58:40.427011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User-Domain-Id: default Jun 10 12:58:40.427011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User-Domain-Name: Default Jun 10 12:58:40.427011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User-Id: 05ecde7b5ffd440ea8bdbb9c0c6418c1 Jun 10 12:58:40.427011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User-Name: admin Jun 10 12:58:40.427011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: method: > Jun 10 12:58:40.427011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: exception: Flavor 42 could not be found. Jun 10 12:58:40.427011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: action_args: {'id': '42'} Jun 10 12:58:40.427011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 12:58:40.427011 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 12:58:40.428138 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:40.429521 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-811c2117-a73c-490c-a794-f50b98887bcc admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103726) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 12:58:40.430777 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-811c2117-a73c-490c-a794-f50b98887bcc admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.014465 Jun 10 12:58:40.430777 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 22/42] 10.222.0.25 () {60 vars in 1190 bytes} [Fri Jun 10 12:58:40 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jun 10 12:58:41.277218 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-4d50fdd7-578d-41ff-aaff-cd01c2798429 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:41.278233 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-4d50fdd7-578d-41ff-aaff-cd01c2798429 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 Jun 10 12:58:41.278629 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 21/43] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Jun 10 12:58:41 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:41.466736 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-c2f10395-8549-4259-b041-572b8b3bbeef admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:41.473067 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.wsgi [None req-c2f10395-8549-4259-b041-572b8b3bbeef admin admin] HTTP exception thrown: Flavor 84 could not be found. Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-c2f10395-8549-4259-b041-572b8b3bbeef admin admin] Request method failure captured: Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Accept: application/json Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Accept-Encoding: gzip, deflate Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Connection: keep-alive Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Content-Length: 0 Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Host: 10.222.0.25 Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Openstack-Api-Version: compute 2.90 Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Openstack-System-Scope: None Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: User-Agent: python-novaclient Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Auth-Token: gAAAAABio0ABaU4cA7fQ5RhhVNnmZUbCeEFKvMkj3pN_yRawdow7H-z1lW64LPCU7iiFv8NPeMNE5cMRb-hShOrpkquOMNWnEw8zgRa-mptbbL4Bn2Hql6Ukp-q9EGFehGrrBZeLVN99Zd3A5J5gmXDQzhAd7T9FxJLmCpl_nPgib_7pqH5I9pE Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Domain-Id: None Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Domain-Name: None Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Identity-Status: Confirmed Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Is-Admin-Project: True Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Openstack-Nova-Api-Version: 2.90 Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Project-Domain-Id: default Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Project-Domain-Name: Default Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Project-Id: 96340edb86a840a88d51d74872a27bac Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Project-Name: admin Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Role: reader,member,admin Jun 10 12:58:41.474552 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Roles: reader,member,admin Jun 10 12:58:41.476090 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/96340edb86a840a88d51d74872a27bac"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/96340edb86a840a88d51d74872a27bac"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_96340edb86a840a88d51d74872a27bac", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/96340edb86a840a88d51d74872a27bac"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 12:58:41.476090 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Tenant: admin Jun 10 12:58:41.476090 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Tenant-Id: 96340edb86a840a88d51d74872a27bac Jun 10 12:58:41.476090 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Tenant-Name: admin Jun 10 12:58:41.476090 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User: admin Jun 10 12:58:41.476090 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User-Domain-Id: default Jun 10 12:58:41.476090 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User-Domain-Name: Default Jun 10 12:58:41.476090 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User-Id: 05ecde7b5ffd440ea8bdbb9c0c6418c1 Jun 10 12:58:41.476090 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User-Name: admin Jun 10 12:58:41.476090 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: method: > Jun 10 12:58:41.476090 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: exception: Flavor 84 could not be found. Jun 10 12:58:41.476090 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: action_args: {'id': '84'} Jun 10 12:58:41.476090 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 12:58:41.476090 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 12:58:41.482922 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:41.484596 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-c2f10395-8549-4259-b041-572b8b3bbeef admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103726) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 12:58:41.485819 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-c2f10395-8549-4259-b041-572b8b3bbeef admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.061104 Jun 10 12:58:41.485819 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 23/44] 10.222.0.25 () {60 vars in 1190 bytes} [Fri Jun 10 12:58:41 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jun 10 12:58:41.486097 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-da37f4b7-680c-4df5-ad73-5a69d25da83a admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:41.501570 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-da37f4b7-680c-4df5-ad73-5a69d25da83a admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.019635 Jun 10 12:58:41.502039 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 22/45] 10.222.0.25 () {60 vars in 1210 bytes} [Fri Jun 10 12:58:41 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 12:58:41.511676 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-6dbd0a8f-f06f-4ba8-8fed-5e5f899cd105 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:41.525749 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-6dbd0a8f-f06f-4ba8-8fed-5e5f899cd105 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017701 Jun 10 12:58:41.526273 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 24/46] 10.222.0.25 () {60 vars in 1210 bytes} [Fri Jun 10 12:58:41 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 12:58:41.535721 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-2dbf23ec-a577-493c-a92c-c9007f160fab admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:41.541076 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.wsgi [None req-2dbf23ec-a577-493c-a92c-c9007f160fab admin admin] HTTP exception thrown: Flavor 84 could not be found. Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-2dbf23ec-a577-493c-a92c-c9007f160fab admin admin] Request method failure captured: Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Accept: application/json Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Accept-Encoding: gzip, deflate Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Connection: keep-alive Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Content-Length: 0 Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Host: 10.222.0.25 Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Openstack-Api-Version: compute 2.90 Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Openstack-System-Scope: None Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: User-Agent: python-novaclient Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Auth-Token: gAAAAABio0ABaU4cA7fQ5RhhVNnmZUbCeEFKvMkj3pN_yRawdow7H-z1lW64LPCU7iiFv8NPeMNE5cMRb-hShOrpkquOMNWnEw8zgRa-mptbbL4Bn2Hql6Ukp-q9EGFehGrrBZeLVN99Zd3A5J5gmXDQzhAd7T9FxJLmCpl_nPgib_7pqH5I9pE Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Domain-Id: None Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Domain-Name: None Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Identity-Status: Confirmed Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Is-Admin-Project: True Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Openstack-Nova-Api-Version: 2.90 Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Project-Domain-Id: default Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Project-Domain-Name: Default Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Project-Id: 96340edb86a840a88d51d74872a27bac Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Project-Name: admin Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Role: reader,member,admin Jun 10 12:58:41.542593 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Roles: reader,member,admin Jun 10 12:58:41.544070 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/96340edb86a840a88d51d74872a27bac"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/96340edb86a840a88d51d74872a27bac"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_96340edb86a840a88d51d74872a27bac", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/96340edb86a840a88d51d74872a27bac"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 12:58:41.544070 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Tenant: admin Jun 10 12:58:41.544070 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Tenant-Id: 96340edb86a840a88d51d74872a27bac Jun 10 12:58:41.544070 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Tenant-Name: admin Jun 10 12:58:41.544070 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User: admin Jun 10 12:58:41.544070 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User-Domain-Id: default Jun 10 12:58:41.544070 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User-Domain-Name: Default Jun 10 12:58:41.544070 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User-Id: 05ecde7b5ffd440ea8bdbb9c0c6418c1 Jun 10 12:58:41.544070 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User-Name: admin Jun 10 12:58:41.544070 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: method: > Jun 10 12:58:41.544070 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: exception: Flavor 84 could not be found. Jun 10 12:58:41.544070 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: action_args: {'id': '84'} Jun 10 12:58:41.544070 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 12:58:41.544070 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 12:58:41.545078 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:41.547279 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-2dbf23ec-a577-493c-a92c-c9007f160fab admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103725) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 12:58:41.556732 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-2dbf23ec-a577-493c-a92c-c9007f160fab admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.011727 Jun 10 12:58:41.556732 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 23/47] 10.222.0.25 () {60 vars in 1190 bytes} [Fri Jun 10 12:58:41 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jun 10 12:58:42.413649 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-6dbd0a8f-f06f-4ba8-8fed-5e5f899cd105 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:42.414651 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-6dbd0a8f-f06f-4ba8-8fed-5e5f899cd105 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001433 Jun 10 12:58:42.415045 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 25/48] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Jun 10 12:58:42 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:42.601073 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-d84383b6-859a-4ea5-81fc-ef3018da842a admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:42.606840 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.wsgi [None req-d84383b6-859a-4ea5-81fc-ef3018da842a admin admin] HTTP exception thrown: Flavor 451 could not be found. Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-d84383b6-859a-4ea5-81fc-ef3018da842a admin admin] Request method failure captured: Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Accept: application/json Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Accept-Encoding: gzip, deflate Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Connection: keep-alive Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Content-Length: 0 Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Host: 10.222.0.25 Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Openstack-Api-Version: compute 2.90 Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Openstack-System-Scope: None Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: User-Agent: python-novaclient Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Auth-Token: gAAAAABio0AC_IQaJvXdjzElhbOSesHtGfU2VjKuLDOkvNOWROP0ap6tTVMQJhwu-qLE46--3TPGTfr93ZPFN_-FW5sROXt2XGWXRZ1yHfl9gH_RoAjQkzE99YcNMsEjJJ57HhK1jzX5QYDkRCLsy16H8CXTco3EkwFeeDvqxHjN1dQ_qDQb6wM Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Domain-Id: None Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Domain-Name: None Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Identity-Status: Confirmed Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Is-Admin-Project: True Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Openstack-Nova-Api-Version: 2.90 Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Project-Domain-Id: default Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Project-Domain-Name: Default Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Project-Id: 96340edb86a840a88d51d74872a27bac Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Project-Name: admin Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Role: reader,member,admin Jun 10 12:58:42.608335 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Roles: reader,member,admin Jun 10 12:58:42.610419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/96340edb86a840a88d51d74872a27bac"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/96340edb86a840a88d51d74872a27bac"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_96340edb86a840a88d51d74872a27bac", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/96340edb86a840a88d51d74872a27bac"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 12:58:42.610419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Tenant: admin Jun 10 12:58:42.610419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Tenant-Id: 96340edb86a840a88d51d74872a27bac Jun 10 12:58:42.610419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-Tenant-Name: admin Jun 10 12:58:42.610419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User: admin Jun 10 12:58:42.610419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User-Domain-Id: default Jun 10 12:58:42.610419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User-Domain-Name: Default Jun 10 12:58:42.610419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User-Id: 05ecde7b5ffd440ea8bdbb9c0c6418c1 Jun 10 12:58:42.610419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: X-User-Name: admin Jun 10 12:58:42.610419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: method: > Jun 10 12:58:42.610419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: exception: Flavor 451 could not be found. Jun 10 12:58:42.610419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: action_args: {'id': '451'} Jun 10 12:58:42.610419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 12:58:42.610419 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 12:58:42.611488 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: ERROR nova.api.openstack.wsgi Jun 10 12:58:42.612885 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-d84383b6-859a-4ea5-81fc-ef3018da842a admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103725) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 12:58:42.614260 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-d84383b6-859a-4ea5-81fc-ef3018da842a admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.058620 Jun 10 12:58:42.614260 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 24/49] 10.222.0.25 () {60 vars in 1193 bytes} [Fri Jun 10 12:58:42 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jun 10 12:58:42.619691 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-d4f23e85-adb6-4570-aa6b-e7830d76d888 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:42.636881 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-d4f23e85-adb6-4570-aa6b-e7830d76d888 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.020686 Jun 10 12:58:42.637412 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 26/50] 10.222.0.25 () {60 vars in 1210 bytes} [Fri Jun 10 12:58:42 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 12:58:42.646693 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-75278f90-dd4c-4143-8088-c82df77797d5 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:42.660202 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-75278f90-dd4c-4143-8088-c82df77797d5 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017086 Jun 10 12:58:42.660748 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 25/51] 10.222.0.25 () {60 vars in 1210 bytes} [Fri Jun 10 12:58:42 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 12:58:42.669544 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-b278bf9d-6f42-4b43-8407-ca92ce70931e admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:42.675250 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.wsgi [None req-b278bf9d-6f42-4b43-8407-ca92ce70931e admin admin] HTTP exception thrown: Flavor 451 could not be found. Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-b278bf9d-6f42-4b43-8407-ca92ce70931e admin admin] Request method failure captured: Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Accept: application/json Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Accept-Encoding: gzip, deflate Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Connection: keep-alive Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Content-Length: 0 Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Host: 10.222.0.25 Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Openstack-Api-Version: compute 2.90 Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Openstack-System-Scope: None Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: User-Agent: python-novaclient Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Auth-Token: gAAAAABio0AC_IQaJvXdjzElhbOSesHtGfU2VjKuLDOkvNOWROP0ap6tTVMQJhwu-qLE46--3TPGTfr93ZPFN_-FW5sROXt2XGWXRZ1yHfl9gH_RoAjQkzE99YcNMsEjJJ57HhK1jzX5QYDkRCLsy16H8CXTco3EkwFeeDvqxHjN1dQ_qDQb6wM Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Domain-Id: None Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Domain-Name: None Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Identity-Status: Confirmed Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Is-Admin-Project: True Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Openstack-Nova-Api-Version: 2.90 Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Project-Domain-Id: default Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Project-Domain-Name: Default Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Project-Id: 96340edb86a840a88d51d74872a27bac Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Project-Name: admin Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Role: reader,member,admin Jun 10 12:58:42.676719 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Roles: reader,member,admin Jun 10 12:58:42.678275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/96340edb86a840a88d51d74872a27bac"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/96340edb86a840a88d51d74872a27bac"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_96340edb86a840a88d51d74872a27bac", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/96340edb86a840a88d51d74872a27bac"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 12:58:42.678275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Tenant: admin Jun 10 12:58:42.678275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Tenant-Id: 96340edb86a840a88d51d74872a27bac Jun 10 12:58:42.678275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-Tenant-Name: admin Jun 10 12:58:42.678275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User: admin Jun 10 12:58:42.678275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User-Domain-Id: default Jun 10 12:58:42.678275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User-Domain-Name: Default Jun 10 12:58:42.678275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User-Id: 05ecde7b5ffd440ea8bdbb9c0c6418c1 Jun 10 12:58:42.678275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: X-User-Name: admin Jun 10 12:58:42.678275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: method: > Jun 10 12:58:42.678275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: exception: Flavor 451 could not be found. Jun 10 12:58:42.678275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: action_args: {'id': '451'} Jun 10 12:58:42.678275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 12:58:42.678275 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 12:58:42.679291 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: ERROR nova.api.openstack.wsgi Jun 10 12:58:42.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-b278bf9d-6f42-4b43-8407-ca92ce70931e admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103726) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 12:58:42.681919 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-b278bf9d-6f42-4b43-8407-ca92ce70931e admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.011988 Jun 10 12:58:42.681919 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 27/52] 10.222.0.25 () {60 vars in 1193 bytes} [Fri Jun 10 12:58:42 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jun 10 12:58:43.531789 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-75278f90-dd4c-4143-8088-c82df77797d5 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:43.532854 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-75278f90-dd4c-4143-8088-c82df77797d5 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002063 Jun 10 12:58:43.533255 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 26/53] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Jun 10 12:58:43 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:43.720937 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-131c5d0b-77f7-4fed-bcfa-9ed3b2a28a87 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:43.818655 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-131c5d0b-77f7-4fed-bcfa-9ed3b2a28a87 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.148365 Jun 10 12:58:43.819193 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 28/54] 10.222.0.25 () {64 vars in 1235 bytes} [Fri Jun 10 12:58:43 2022] POST /compute/v2.1/flavors => generated 404 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 12:58:44.715562 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-75278f90-dd4c-4143-8088-c82df77797d5 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:44.716600 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-75278f90-dd4c-4143-8088-c82df77797d5 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001441 Jun 10 12:58:44.717034 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 27/55] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Jun 10 12:58:44 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:44.905260 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-6ec5ea1e-929d-4c9b-ab38-990f08e8dde8 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:44.975290 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-6ec5ea1e-929d-4c9b-ab38-990f08e8dde8 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.121073 Jun 10 12:58:44.975834 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 29/56] 10.222.0.25 () {64 vars in 1235 bytes} [Fri Jun 10 12:58:44 2022] POST /compute/v2.1/flavors => generated 405 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 12:58:45.855676 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-75278f90-dd4c-4143-8088-c82df77797d5 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:45.856680 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-75278f90-dd4c-4143-8088-c82df77797d5 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001403 Jun 10 12:58:45.857114 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 28/57] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Jun 10 12:58:45 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:46.049142 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-890cb625-0807-41da-bf43-df745fa1733c admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:46.130159 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-890cb625-0807-41da-bf43-df745fa1733c admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.132276 Jun 10 12:58:46.130637 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 30/58] 10.222.0.25 () {64 vars in 1235 bytes} [Fri Jun 10 12:58:45 2022] POST /compute/v2.1/flavors => generated 407 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 12:58:47.004331 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-75278f90-dd4c-4143-8088-c82df77797d5 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:47.005443 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-75278f90-dd4c-4143-8088-c82df77797d5 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001519 Jun 10 12:58:47.005833 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 29/59] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Jun 10 12:58:47 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:47.194567 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-32f7ea29-aa68-4d97-b20c-925eb68ec958 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 12:58:47.342445 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-32f7ea29-aa68-4d97-b20c-925eb68ec958 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.199009 Jun 10 12:58:47.343611 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 31/60] 10.222.0.25 () {64 vars in 1235 bytes} [Fri Jun 10 12:58:47 2022] POST /compute/v2.1/flavors => generated 409 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 12:58:48.210470 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-75278f90-dd4c-4143-8088-c82df77797d5 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:48.211551 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-75278f90-dd4c-4143-8088-c82df77797d5 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001480 Jun 10 12:58:48.211952 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 30/61] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Jun 10 12:58:48 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:58:48.401269 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-d144eb9b-299b-4395-b57f-022866b93223 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:58:48.420621 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-d144eb9b-299b-4395-b57f-022866b93223 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.070223 Jun 10 12:58:48.421669 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 32/62] 10.222.0.25 () {60 vars in 1202 bytes} [Fri Jun 10 12:58:48 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 12:59:36.130632 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-75278f90-dd4c-4143-8088-c82df77797d5 admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:59:36.131635 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-75278f90-dd4c-4143-8088-c82df77797d5 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001678 Jun 10 12:59:36.132029 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 31/63] 10.222.0.25 () {54 vars in 930 bytes} [Fri Jun 10 12:59:36 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 12:59:36.142767 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG dbcounter [-] [103726] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=103726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:59:36.189745 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-5e299b2b-b0bb-47fb-a2c9-a67dca3fb9a7 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:59:36.209233 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-5e299b2b-b0bb-47fb-a2c9-a67dca3fb9a7 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.070823 Jun 10 12:59:36.209771 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 33/64] 10.222.0.25 () {60 vars in 1291 bytes} [Fri Jun 10 12:59:36 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 12:59:36.232553 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-625f2efd-6360-4f9c-a3c9-1c48126fbcba admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:59:36.235938 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG dbcounter [-] [103725] Writing DB stats nova_api:SELECT=14 {{(pid=103725) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:59:36.240828 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-625f2efd-6360-4f9c-a3c9-1c48126fbcba admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011845 Jun 10 12:59:36.241340 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 32/65] 10.222.0.25 () {60 vars in 1282 bytes} [Fri Jun 10 12:59:36 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 12:59:36.253407 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-3da570e2-0b8d-45de-a222-8880947c7828 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:59:36.261530 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-3da570e2-0b8d-45de-a222-8880947c7828 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011978 Jun 10 12:59:36.262641 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 34/66] 10.222.0.25 () {60 vars in 1285 bytes} [Fri Jun 10 12:59:36 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 12:59:36.273817 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-1f5c780d-aa73-4e55-a515-a626e0221f2b admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:59:36.281100 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-1f5c780d-aa73-4e55-a515-a626e0221f2b admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010992 Jun 10 12:59:36.281613 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 33/67] 10.222.0.25 () {60 vars in 1285 bytes} [Fri Jun 10 12:59:36 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 12:59:36.292394 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-9b764b1d-66bf-4ead-9170-0de308c6a611 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 12:59:36.299752 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-9b764b1d-66bf-4ead-9170-0de308c6a611 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010769 Jun 10 12:59:36.300261 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 35/68] 10.222.0.25 () {60 vars in 1282 bytes} [Fri Jun 10 12:59:36 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:03:01.244320 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-1f5c780d-aa73-4e55-a515-a626e0221f2b admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:03:01.245365 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-1f5c780d-aa73-4e55-a515-a626e0221f2b admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001739 Jun 10 13:03:01.246026 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 34/69] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Jun 10 13:03:01 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:03:01.400553 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG dbcounter [-] [103726] Writing DB stats nova_api:SELECT=6 {{(pid=103726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:01.511018 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-bce9d6ba-ceac-4a11-9a81-e1946c6c2995 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:03:01.512181 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 10 13:03:01.512181 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: warnings.warn(msg) Jun 10 13:03:01.513409 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-bce9d6ba-ceac-4a11-9a81-e1946c6c2995 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:03:01.513852 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-bce9d6ba-ceac-4a11-9a81-e1946c6c2995 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:03:01.514623 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-bce9d6ba-ceac-4a11-9a81-e1946c6c2995 admin admin] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:03:01.515051 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-bce9d6ba-ceac-4a11-9a81-e1946c6c2995 admin admin] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:03:01.548261 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-bce9d6ba-ceac-4a11-9a81-e1946c6c2995 admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.149136 Jun 10 13:03:01.548847 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 36/70] 10.222.0.25 () {60 vars in 1193 bytes} [Fri Jun 10 13:03:01 2022] GET /compute/v2.1/os-services => generated 791 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:03:02.410640 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-1f5c780d-aa73-4e55-a515-a626e0221f2b admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:03:02.413573 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-1f5c780d-aa73-4e55-a515-a626e0221f2b admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003323 Jun 10 13:03:02.413977 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 35/71] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Jun 10 13:03:02 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:03:02.606585 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-fc6e1300-e79a-4775-8468-eeb925f6404e admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:03:02.607742 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 10 13:03:02.607742 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: warnings.warn(msg) Jun 10 13:03:02.609048 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-fc6e1300-e79a-4775-8468-eeb925f6404e admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:03:02.609484 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-fc6e1300-e79a-4775-8468-eeb925f6404e admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:03:02.610279 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-fc6e1300-e79a-4775-8468-eeb925f6404e admin admin] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:03:02.611016 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-fc6e1300-e79a-4775-8468-eeb925f6404e admin admin] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:03:02.630851 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-fc6e1300-e79a-4775-8468-eeb925f6404e admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.075331 Jun 10 13:03:02.631390 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 37/72] 10.222.0.25 () {60 vars in 1193 bytes} [Fri Jun 10 13:03:02 2022] GET /compute/v2.1/os-services => generated 791 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:03:13.938615 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-1f5c780d-aa73-4e55-a515-a626e0221f2b admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:03:13.939704 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-1f5c780d-aa73-4e55-a515-a626e0221f2b admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001487 Jun 10 13:03:13.940111 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 36/73] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Jun 10 13:03:13 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:03:14.085496 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG dbcounter [-] [103726] Writing DB stats nova_cell0:SELECT=4 {{(pid=103726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:14.086218 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG dbcounter [-] [103726] Writing DB stats nova_cell1:SELECT=4 {{(pid=103726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:14.086830 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG dbcounter [-] [103726] Writing DB stats nova_api:SELECT=8 {{(pid=103726) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:14.137912 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-864e1efc-275f-467a-9434-ff8d622ae705 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:03:14.139162 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 10 13:03:14.139162 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: warnings.warn(msg) Jun 10 13:03:14.140728 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-864e1efc-275f-467a-9434-ff8d622ae705 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:03:14.140728 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-864e1efc-275f-467a-9434-ff8d622ae705 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:03:14.141574 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-864e1efc-275f-467a-9434-ff8d622ae705 admin admin] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:03:14.141997 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-864e1efc-275f-467a-9434-ff8d622ae705 admin admin] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:03:14.167446 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-864e1efc-275f-467a-9434-ff8d622ae705 admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.087129 Jun 10 13:03:14.169569 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 38/74] 10.222.0.25 () {60 vars in 1193 bytes} [Fri Jun 10 13:03:14 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:03:15.057402 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: DEBUG nova.api.openstack.wsgi [None req-1f5c780d-aa73-4e55-a515-a626e0221f2b admin admin] Calling method '>' {{(pid=103725) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:03:15.058417 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: INFO nova.api.openstack.requestlog [None req-1f5c780d-aa73-4e55-a515-a626e0221f2b admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001443 Jun 10 13:03:15.058817 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: [pid: 103725|app: 0|req: 37/75] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Jun 10 13:03:15 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:03:15.249956 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG nova.api.openstack.wsgi [None req-b0a6e0d2-f632-4060-8534-c5c910a89249 admin admin] Calling method '>' {{(pid=103726) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:03:15.251127 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 10 13:03:15.251127 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: warnings.warn(msg) Jun 10 13:03:15.252325 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-b0a6e0d2-f632-4060-8534-c5c910a89249 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:03:15.252813 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-b0a6e0d2-f632-4060-8534-c5c910a89249 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:03:15.253613 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-b0a6e0d2-f632-4060-8534-c5c910a89249 admin admin] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:03:15.254066 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: DEBUG oslo_concurrency.lockutils [None req-b0a6e0d2-f632-4060-8534-c5c910a89249 admin admin] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103726) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:03:15.275769 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: INFO nova.api.openstack.requestlog [None req-b0a6e0d2-f632-4060-8534-c5c910a89249 admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.077499 Jun 10 13:03:15.276254 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: [pid: 103726|app: 0|req: 39/76] 10.222.0.25 () {60 vars in 1193 bytes} [Fri Jun 10 13:03:15 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:04:11.939097 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Jun 10 13:04:11.939636 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: Fri Jun 10 13:04:11 2022 - graceful shutdown triggered... Jun 10 13:04:11.940048 dv-cinder-master-3-iscsi devstack@n-api.service[103725]: Gracefully killing worker 1 (pid: 103725)... Jun 10 13:04:11.940526 dv-cinder-master-3-iscsi devstack@n-api.service[103726]: Gracefully killing worker 2 (pid: 103726)... Jun 10 13:04:12.940559 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: worker 1 buried after 1 seconds Jun 10 13:04:12.940559 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: worker 2 buried after 1 seconds Jun 10 13:04:12.941226 dv-cinder-master-3-iscsi devstack@n-api.service[103724]: goodbye to uWSGI. Jun 10 13:04:12.942171 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Jun 10 13:04:12.942724 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Jun 10 13:04:12.945669 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Jun 10 13:04:12.955374 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jun 10 13:04:12.957292 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 10 13:04:12 2022] *** Jun 10 13:04:12.957292 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 10 13:04:12.957292 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 10 13:04:12.957292 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: nodename: dv-cinder-master-3-iscsi Jun 10 13:04:12.957292 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: machine: x86_64 Jun 10 13:04:12.957292 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: clock source: unix Jun 10 13:04:12.957292 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: pcre jit disabled Jun 10 13:04:12.957292 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: detected number of CPU cores: 4 Jun 10 13:04:12.957292 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: current working directory: / Jun 10 13:04:12.957292 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: detected binary path: /usr/bin/uwsgi-core Jun 10 13:04:12.958006 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: your processes number limit is 47680 Jun 10 13:04:12.958006 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: your memory page size is 4096 bytes Jun 10 13:04:12.958006 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: detected max file descriptor number: 2048 Jun 10 13:04:12.958006 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: lock engine: pthread robust mutexes Jun 10 13:04:12.958006 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: thunder lock: enabled Jun 10 13:04:12.958006 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jun 10 13:04:12.958006 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 10 13:04:13.002457 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: Python main interpreter initialized at 0x55acfea0d9c0 Jun 10 13:04:13.002457 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: python threads support enabled Jun 10 13:04:13.002457 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: your server socket listen backlog is limited to 100 connections Jun 10 13:04:13.002457 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: your mercy for graceful operations on workers is 90 seconds Jun 10 13:04:13.002918 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: mapped 403077 bytes (393 KB) for 2 cores Jun 10 13:04:13.002918 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: *** Operational MODE: preforking *** Jun 10 13:04:13.003201 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: *** uWSGI is running in multiple interpreter mode *** Jun 10 13:04:13.003201 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: spawned uWSGI master process (pid: 118076) Jun 10 13:04:13.003417 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Jun 10 13:04:13.004564 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: spawned uWSGI worker 1 (pid: 118092, cores: 1) Jun 10 13:04:13.004564 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: spawned uWSGI worker 2 (pid: 118093, cores: 1) Jun 10 13:04:13.004564 dv-cinder-master-3-iscsi devstack@n-api.service[118076]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 10 13:04:13.670032 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jun 10 13:04:13.673391 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jun 10 13:04:14.915687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 10 13:04:14.915687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: from cryptography.utils import int_from_bytes Jun 10 13:04:14.916326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 10 13:04:14.916326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: from cryptography.utils import int_from_bytes Jun 10 13:04:14.925491 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 10 13:04:14.925491 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: from cryptography.utils import int_from_bytes Jun 10 13:04:14.925953 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 10 13:04:14.925953 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: from cryptography.utils import int_from_bytes Jun 10 13:04:17.336317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 10 13:04:17.336670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 10 13:04:17.337141 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 10 13:04:17.337509 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 10 13:04:17.337771 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 10 13:04:17.338148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.338477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.338801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.339129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.339460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.339770 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.340090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.340409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.340739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.341101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.341419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.341742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.342065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.342372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.342683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.343000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.343302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.343621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.344487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.345021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.345822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.346401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.346885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.347235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.347553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.347869 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.348199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.348508 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.349016 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.349340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.349739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.350228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 10 13:04:17.350627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.350713 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 10 13:04:17.350800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.350880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.351013 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 10 13:04:17.351260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.351587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.351701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 10 13:04:17.351904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.352018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 10 13:04:17.352210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.352334 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.352665 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.352892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.353084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.353255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.353368 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.353449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.353674 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.353844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.354010 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.354164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.354309 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.354489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.354656 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.354813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.354971 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.355128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.355284 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.355446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.355589 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.355762 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.355920 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.356071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.356226 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.356393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.356393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.356668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.356849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.357007 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.357158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.357329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.357499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.357651 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.357803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.357962 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.358112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.358256 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.358466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.358466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.358847 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.359043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.359215 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.359426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.360683 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.360916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.361066 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.361456 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.361624 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.361782 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.361927 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.362069 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.362212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.362379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.362535 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.362680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.362822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.362979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.363131 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.363284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.363428 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.363580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.363731 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.363875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.364051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.364195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.364355 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.364506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.364660 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.364838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.364996 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.365148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.365300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.365453 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.365636 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.365802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.365958 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.366112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.366112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.366409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.366553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.366707 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.366858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.367000 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.367401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.367555 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.367697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.367852 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.368003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.368145 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.368302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.368461 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.368617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.368778 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.368932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.369088 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.369240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.369394 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.369546 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.369697 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.369840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.370009 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.370962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.371118 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.371283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.371436 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.371580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.371734 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.371888 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.372039 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.372196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.372348 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.372730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.373149 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.373304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.373457 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.373627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.373770 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.374099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.374245 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.374400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.374554 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.374742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.374906 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.375050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.375201 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.375358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.375504 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.375668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.375812 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.375961 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.376111 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.376264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.376414 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.376566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.376709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.376886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.377044 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.377202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.377353 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.377495 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.377661 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.377812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.377955 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.378107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.378258 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.378410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.378601 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.378750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.378893 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.379042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.379196 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.379346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.379577 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.379733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.379876 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.380035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.380187 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.380359 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.381190 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.381347 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.381501 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.381702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.381859 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.382012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.382163 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.382318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.382470 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.382613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.382754 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.382904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.382904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.383191 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.383342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.383496 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.383496 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.383790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.383943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.384095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.384247 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.384399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.384399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.384689 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.384865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.385020 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.385169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.385312 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.385472 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.385628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.385780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.385924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.386066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.386226 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.386379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.386529 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.386673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.386822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.386822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.387099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.387251 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.387393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.387542 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.387691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.387841 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.387983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.388150 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.388294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.388437 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.388587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.388738 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.388958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.389104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.389245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.389394 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.389544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.389695 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.389847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.390001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.390150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.390306 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.390450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.390617 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.391824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.391978 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.392122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.392279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.392423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.392566 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.392717 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.392897 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.393055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.393211 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.393356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.393505 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.393648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.393799 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.393967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.394113 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.394256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.394398 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.394547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.394691 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.394842 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.394995 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.395138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.395289 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.395432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.395575 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.395731 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.395874 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.395874 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.396151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.396293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.396434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.396586 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.396740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.396998 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.397156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.397307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.397460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.397604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.397753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.397905 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.398048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.398199 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.398357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.398515 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.398668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.398833 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.398977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.399127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.399278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.399438 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.399581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.399723 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 13:04:17.399865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.400014 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.400163 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.400316 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.400461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.400611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.400780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.400941 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.401094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.401247 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.402242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.402390 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.402534 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.402685 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.402842 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.402994 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.403138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.403289 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.403452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.403604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.403761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.403923 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.404068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.404219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.404370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.404521 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.404674 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.404918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.405168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.405317 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.405460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.405623 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.405776 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.405928 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.406071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.406223 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.406418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.406418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.406716 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.406859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.406859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.407135 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.407279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.407498 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.407663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.407816 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.407959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.408101 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.408243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.408399 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.408549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.408549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.408865 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.409013 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.409166 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.409166 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.409455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.409615 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.409767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.409918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.410070 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.410214 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.410364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.410516 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.410702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.410861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.411014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.411166 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.411318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.411318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.412265 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.412417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.412573 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.412717 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.412884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.413041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.413193 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.413336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.413489 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.413632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.413807 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.413964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.414119 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.414276 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.414429 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.414579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.414722 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.414871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.415015 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.415166 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.415310 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.415452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.415601 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.415752 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.415906 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.416069 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.416220 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.416363 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.416513 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.416663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.416827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.416993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.417140 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.417290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.417475 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.417633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.417788 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.417939 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.418095 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.418255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.418255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.418539 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.418697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.418847 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.418990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.419141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.419284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.419426 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.419577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.419728 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.419900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.420051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.420194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.420337 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.420491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.420635 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.420864 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.421017 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.421174 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.421351 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.421587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.421587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.422759 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.422925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.422925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.423206 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.423357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.423357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.423637 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.423821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.423976 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.424134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.424134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.424423 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.424566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.424566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.424888 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.425047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.425047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.425328 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.425472 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.425472 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.425742 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.425893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.425893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.426179 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.426323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.426473 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.426631 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.426631 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.426924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.427075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.427075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.427352 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.427509 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.427661 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.427811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.427953 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.428095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.428244 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.428400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.428560 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.428705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.428705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.429062 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.429213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.429394 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.429596 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.429744 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.429887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.430037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.430189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.430332 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.430473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.430634 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.430785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.430927 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.431083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.431227 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.431368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.431518 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.431668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.431832 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.431989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.432141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.432291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.432442 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.432916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.433074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.433231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.433383 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.433533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.433683 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.433834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.433996 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.434148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.434291 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.434440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.434591 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.434740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.434908 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.435052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.435193 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.435335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.435484 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.435633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.435791 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.435942 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.436092 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.436249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.436249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.436528 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.436680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.436897 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.437052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.437196 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.437355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.437628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.438290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.438451 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.438609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.438765 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.438914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.439078 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.439235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.439395 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.439544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.439705 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.439863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.440026 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.440183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.440333 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.440481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.440630 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.440804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.440974 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.441124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.441276 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.441441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.441590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.441745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.441905 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.442055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.442212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.442361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.442517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.442973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.443134 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.443284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.443433 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.443588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.443758 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.443930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.444083 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.444238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.444238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.444531 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.444687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.444890 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.445045 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.445200 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.445349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.445497 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.445652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.445864 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.446028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.446178 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.446326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.446482 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.446644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.446793 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.446942 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.447097 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.447254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.447411 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.447573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.447731 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.447891 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.448048 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.448196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.448352 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.448512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.448669 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.448848 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.449019 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.449170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.449324 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.449482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.449639 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.449639 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.449941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.450105 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.450263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.450419 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.450567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.450723 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.450878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.451035 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.451198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.451354 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.451503 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.451652 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.451816 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.451972 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.452129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.452279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.452427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.452575 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.452754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.453042 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.453869 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.454039 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.454196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.454357 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.454515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.454687 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.454845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.455005 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.455156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.455318 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.455468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.455625 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.455784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.455982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.456139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.456297 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.456446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.456594 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.456791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.456965 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.457124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.457124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.457417 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.457573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.457730 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.457878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.458041 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.458191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.458356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.458516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.458676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.458825 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.458984 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.459140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.459289 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.459445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.459602 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.459762 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.459921 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.460073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.460237 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.460394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.460544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.460712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.460887 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.461038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.461199 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.461355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.461505 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.461662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.461813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.461961 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.462109 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.462266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.462423 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.462617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.462786 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.463028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.463188 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.463338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.463497 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.463654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.463803 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.463971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.464123 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.464272 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.464435 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.464593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.464743 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.464933 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.465103 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.465254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.465418 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.465577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.465734 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.465885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.466033 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.466181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.466340 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.466490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.466646 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.466813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.466970 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.467129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.467286 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.467436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.467436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.467728 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.467884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.468048 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.468207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.468207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.468500 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.468656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.468920 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.469589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.469748 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.469898 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.470046 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.470201 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.470352 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.470579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.470740 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.470891 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.471039 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.471196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.471364 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.471532 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.471692 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.471841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.472004 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.472155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.472305 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.472461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.472611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.472790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.472966 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.473151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.473302 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.473459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.473620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.473770 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.473945 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.474106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.474264 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.474414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.474414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.474716 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.474910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.475182 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.475182 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.475642 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.475642 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.476151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.476395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.476659 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.477024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.477228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.477383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.477615 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.477769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.477964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.478129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.478280 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.478429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.478600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.478750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.478900 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.479056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.479206 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.479362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.479512 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.479669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.479818 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.479968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.480124 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.480284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.480435 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.480594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.480743 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.480949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.481104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.481265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.481416 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.481572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.481722 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.481870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.482034 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.482193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.482350 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.482506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.482666 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.482823 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.482982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.483132 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.483288 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.483445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.483602 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.483759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.483930 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.484090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.484668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.484882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.485064 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.485223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.485389 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.485540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.485689 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.485846 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.485997 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.486146 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.486310 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.486462 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.486611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.486760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.486910 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.487078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.487228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.487378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.487535 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.487686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.487835 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.488035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.488192 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.488350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.488500 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.488649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.488821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.488990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.489160 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.489310 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.489461 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.489610 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.489768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.489931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.490083 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.490232 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.490388 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.490547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.490712 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.490870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.491021 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.491179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.491328 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.491477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.491477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.491777 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.491935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.492349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.492512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.492664 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.492867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.493032 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.493190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.493347 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.493516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.493668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.493826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.493988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.494138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.494286 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.494443 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.494607 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.494767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.494917 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.495066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.495223 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.495383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.495541 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.495693 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.495850 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.496000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.496158 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.496307 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.496551 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.496713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.496894 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.497047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.497196 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.497356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.497532 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.497690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.497841 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.497997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.498146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.498309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.498468 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.498468 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.498762 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.498976 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.499137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.499302 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.499453 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.499620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.499770 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.499921 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.500138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.500299 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.500451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.500608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.500788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.500788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.501084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.501246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.501397 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.501546 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.501713 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.501869 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.502047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.502280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.502435 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.502654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.502814 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.502970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.503130 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.503281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.503430 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.503586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.503755 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.503958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.504128 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.504128 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.504414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.504563 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.504711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.504916 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.505085 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.505319 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.505474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.505624 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.505780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.505958 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.506117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.506269 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.506425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.506576 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.506725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.506889 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.507040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.507040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.507324 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.507474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.507622 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.507855 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.508019 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.508169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.508326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.508477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.508690 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.508928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.509048 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.509146 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.509238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.509326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.509424 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.509521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.509610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.509610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.509785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.509880 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.509977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.510072 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.510166 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.510260 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.510347 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.510438 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.510534 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.510628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.510716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.510800 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.510884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.510958 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.511039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.511114 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.511192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.511192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.511332 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.511332 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.511481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.511481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.511644 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.511728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.511823 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.511919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.512006 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.512091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.512175 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.512266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.512359 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.512460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.512555 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.512649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.512736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.512885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.512974 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.513065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.513147 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.513219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.513298 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.513379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.513487 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.513591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.513676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.513757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.513857 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.513965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.513965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.514099 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.514177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.514257 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.514330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.514415 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.514496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.514576 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.514655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.514736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.514822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.514896 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.514974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.515047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.515118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.515189 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.515273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.515348 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.515419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.515497 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.515575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.516159 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.516245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.516319 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.516404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.516484 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.516563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.516636 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.516719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.516830 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.516907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.516979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.517063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.517147 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.517231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.517312 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.517391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.517471 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.517542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.517622 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.517701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.517701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.517838 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.517920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.517994 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.518078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.518158 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.518230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.518307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.518386 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.518586 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.518670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.518750 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.518829 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.518901 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.518971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.519050 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.519123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.519200 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.519280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.519361 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.519441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.519527 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.519607 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.519679 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.519749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.519819 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.519899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.519985 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.520058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.520129 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.520207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.520286 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.520454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.520550 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.520636 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.520717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.520820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.520919 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.521016 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.521103 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.521186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.521347 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.521497 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.521663 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.521777 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.521976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.522071 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.522285 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.522378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.522595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.522697 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.522905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.523051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.523219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.523321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.523531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.523633 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.523848 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.523940 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.524166 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.524402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.524521 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.524729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.530593 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.530764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.530868 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.530966 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.531069 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.531166 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.531261 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.531897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.532003 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.532098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.532196 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.532292 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.532387 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.532473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.532559 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.532653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.532740 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.532893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.532985 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.533066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.533138 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.533229 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.533319 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.533412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.533499 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.533592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.533680 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.533817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.533920 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.534055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.534151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.534247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.534345 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.534427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.534509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.534596 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.534682 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.534774 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.534873 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.534949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.535033 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.535125 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.535219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.535305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.535399 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.535493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.535579 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.535663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.535747 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.535845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.535932 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.536016 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.536101 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.536193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.536280 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.536378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.536472 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.536558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.536651 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.536736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.536909 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.537011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.537125 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.537427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.537518 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.537614 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.537717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.537806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.537933 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.538026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.538113 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.538197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.538284 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.538379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.538466 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.538466 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.538627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.538714 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.538808 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.538897 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.538981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.539067 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.539151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.539243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.539339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.539427 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.539512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.539597 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.539688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.539784 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.539872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.539958 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.540042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.540126 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.540210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.540309 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.540403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.540489 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.540574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.540751 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.540871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.541095 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.541194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.541405 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.541498 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.541711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.541804 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.542031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.542173 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.542341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.542444 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.542653 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.542877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.542983 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.543233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.543327 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.543554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.543649 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.543884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.544134 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.544237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.544472 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.544567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.544822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.544937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.545149 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.545233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.545473 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.545598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.545787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.545895 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.546105 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.546198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.546417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.546511 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.546723 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.546826 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.547064 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.547159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.547387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.547502 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.547688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.547839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.548025 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.548147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.548378 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.548493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.548669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.548794 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.549022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.549127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.549331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.549509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.549643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.549739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.549949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.550052 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.550263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.550373 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.550570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.550666 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.550876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.550978 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.551180 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.551289 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.551483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.551587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.551796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.551890 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.552141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.552235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.552450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.552550 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.552757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.552902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.553093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.553190 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.553398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.553492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.553710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.553793 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.554028 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.554137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.554351 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.554488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.554664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.554766 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.554959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.555056 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.555276 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.555378 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.555588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.555693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.555911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.556008 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.556225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.556327 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.556538 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.556759 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.559255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.559388 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.559482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.559570 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.559655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.559755 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.559844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.559936 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.560029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.560116 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.560212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.560305 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.560393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.560485 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.560571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.560656 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.560656 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.560900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.560990 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.561083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.561177 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.561270 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.561368 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.561462 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.561549 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.561642 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.561730 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.561803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.561897 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.562034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.562132 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.562219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.562219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.562385 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.562493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.562616 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.562719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.562970 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.563073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.563296 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.563398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.563624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.563718 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.563935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.564030 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.564245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.564341 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.564559 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.564654 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.564910 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.565031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.565226 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.565322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.565540 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.565643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.565854 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.565958 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.566171 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.566265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.566479 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.566572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.566790 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.566897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.567098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.567190 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.567511 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.567804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.567903 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.568001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.568096 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.568191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.568325 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.568444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.568661 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.568798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.568997 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.569108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.569311 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.569407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.569623 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.569719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.569943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.570037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.570258 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.570355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.570571 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.570667 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.570883 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.570976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.571194 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.571326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.571512 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.571682 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.571824 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.571927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.572141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.572251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.572455 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.572550 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.572798 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.572896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.573104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.573217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.573422 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.573523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.573726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.573827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.574095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.574243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.574400 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.574493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.574713 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.574808 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.575023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.575117 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.575345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.575452 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.575651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.575746 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.575964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.576069 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.576271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.576365 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.576576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.576678 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.576934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.577050 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.577211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.577503 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.577668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.577869 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.577979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.578621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.578770 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.578952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.579051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.579262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.579359 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.579574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.579670 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.579880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.579981 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.580190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.580293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.580499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.580602 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.580830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.580944 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.581138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.581241 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.581445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.581547 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.581751 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.581845 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.582083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.582187 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.582405 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.582505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.582725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.582829 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.583042 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.583137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.583354 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.583448 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.583667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.583801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.583978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.584072 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.584290 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.584384 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.584607 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.584715 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.588014 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.588150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.588150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.588323 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.588444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.588541 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.588628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.588720 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.588862 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.588957 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.589053 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.589140 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.589232 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.589325 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.589418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.589504 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.589597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.589684 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.589775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.589861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.589952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.590037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.590134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.590228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.590323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.590409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.590409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.590577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.590577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.590742 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.590833 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.590918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.591008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.591104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.591197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.591283 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.591364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.591436 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.591513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.591596 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.591797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.591872 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.591976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.592060 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.592145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.592219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.592297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.592377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.592450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.592527 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.592664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.592821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.593126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.593277 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.593441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.593600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.593775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.593944 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.594035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.594130 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.594325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.594484 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.594640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.594735 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.594984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.595078 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.595312 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.595479 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.595620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.595761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.595942 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.596105 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.596251 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.596430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.596577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 10 13:04:17.596727 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.596946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.597141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.597455 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.597772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.598084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.598395 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.598707 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.599024 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.599337 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.599649 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.599966 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.600277 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.600591 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.600925 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.601238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.601552 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.601892 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.602236 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.602560 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.602891 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.603210 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.603540 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.603860 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.604191 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.604514 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.604859 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.605181 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.605521 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.605840 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.606165 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.606483 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.606801 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.607119 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.607435 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.607767 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.608104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.608418 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.608752 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.609097 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.609940 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.610266 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.610586 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.610924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.611243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.611561 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.611886 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.612204 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.612523 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.612869 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.613198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.613532 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.613866 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.614184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.614516 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.614850 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.615207 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.615529 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.615883 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.616225 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.616548 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.616890 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.617460 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 13:04:17.617710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118093) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 10 13:04:17.659506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO dbcounter [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] Registered counter for database nova_api Jun 10 13:04:17.679783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_db.sqlalchemy.engines [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118092) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 13:04:17.681128 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO dbcounter [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] Registered counter for database nova_api Jun 10 13:04:17.686310 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writer thread running {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 13:04:17.700940 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_db.sqlalchemy.engines [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118093) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 13:04:17.707452 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writer thread running {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 13:04:17.744911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.context [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),bf155fd7-c737-40a3-a522-fd4049b0e8da(cell1) {{(pid=118092) load_cells /opt/stack/nova/nova/context.py:464}} Jun 10 13:04:17.749046 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:04:17.750651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:04:17.751443 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:04:17.753429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:04:17.755409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO dbcounter [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] Registered counter for database nova_cell0 Jun 10 13:04:17.759257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO dbcounter [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] Registered counter for database nova_cell1 Jun 10 13:04:17.766019 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.context [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),bf155fd7-c737-40a3-a522-fd4049b0e8da(cell1) {{(pid=118093) load_cells /opt/stack/nova/nova/context.py:464}} Jun 10 13:04:17.766218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_db.sqlalchemy.engines [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118092) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 13:04:17.767504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_db.sqlalchemy.engines [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118092) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 13:04:17.770260 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:04:17.771864 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:04:17.772676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:04:17.773405 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writer thread running {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 13:04:17.774690 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:04:17.775995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writer thread running {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 13:04:17.776703 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO dbcounter [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] Registered counter for database nova_cell0 Jun 10 13:04:17.780611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO dbcounter [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] Registered counter for database nova_cell1 Jun 10 13:04:17.788517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_db.sqlalchemy.engines [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118093) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 13:04:17.790337 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_db.sqlalchemy.engines [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118093) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 13:04:17.795968 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writer thread running {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 13:04:17.801356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writer thread running {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 13:04:17.926577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO dbcounter [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] Registered counter for database nova_cell0 Jun 10 13:04:17.933076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_db.sqlalchemy.engines [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118092) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 13:04:17.937523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writer thread running {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 13:04:17.949466 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO dbcounter [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] Registered counter for database nova_cell0 Jun 10 13:04:17.956474 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_db.sqlalchemy.engines [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118093) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 13:04:17.961492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writer thread running {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 13:04:18.264920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: WARNING keystonemiddleware.auth_token [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jun 10 13:04:18.272751 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: WARNING keystonemiddleware.auth_token [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jun 10 13:04:18.291366 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: WARNING keystonemiddleware.auth_token [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jun 10 13:04:18.299186 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: WARNING keystonemiddleware.auth_token [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jun 10 13:04:18.620158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: WARNING keystonemiddleware.auth_token [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jun 10 13:04:18.626159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: WARNING keystonemiddleware.auth_token [None req-26d1edd5-58df-4f7b-8783-51f4df0cd413 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jun 10 13:04:18.631434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55acfea0d9c0 pid: 118092 (default app) Jun 10 13:04:18.659820 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: WARNING keystonemiddleware.auth_token [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jun 10 13:04:18.665840 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: WARNING keystonemiddleware.auth_token [None req-f07feaa3-cb5e-495c-88fe-0b7ecdc87d14 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jun 10 13:04:18.671128 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55acfea0d9c0 pid: 118093 (default app) Jun 10 13:04:59.922624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=3 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:59.923560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=3 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:59.924548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=3 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:59.925458 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=3 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:01.033501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2140f151-5c5d-4564-8c33-cc154a90c97b tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] Action: 'create', calling method: .version_select at 0x7f93071ec3a0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-158885065", "metadata": {"Type": "work"}}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:05:02.355463 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=3 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:02.375914 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=3 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:02.375914 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=3 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:02.375914 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=3 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:02.866018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-848068801", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "networks": [{"uuid": "74438650-ae76-4f06-b2ba-22d6d4ed3680"}]}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:05:03.761040 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.network.neutron [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] validate_networks() for [('74438650-ae76-4f06-b2ba-22d6d4ed3680', None, None, None, None, None)] {{(pid=118093) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:05:04.730709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2140f151-5c5d-4564-8c33-cc154a90c97b tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 200 len: 352 microversion: 2.1 time: 4.816250 Jun 10 13:05:04.739794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1/1] 10.222.0.25 () {60 vars in 1155 bytes} [Fri Jun 10 13:04:59 2022] POST /compute/v2.1/os-volumes => generated 352 bytes in 4829 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Jun 10 13:05:04.745594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-55f5b7aa-30ef-45b1-893e-80a69a8778f1 tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] Calling method '.version_select at 0x7f93070be280>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:04.837951 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Flavor limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:05:04.839769 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Image limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:05:04.840686 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Flavor pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:05:04.842348 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Image pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:05:04.843867 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:05:04.863050 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:04.863780 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:04.864576 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:04.865306 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:04.901648 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Getting quotas for project 72affe24c05f4972a9a30883d2cbc85f. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:05:04.909097 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Getting quotas for user 43c31865d2d744d6bb809a7048d2c575 and project 72affe24c05f4972a9a30883d2cbc85f. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:05:04.929032 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-55f5b7aa-30ef-45b1-893e-80a69a8778f1 tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/abc42eea-e8f2-451c-ac62-968cd4d8c1aa" status: 200 len: 352 microversion: 2.1 time: 0.187035 Jun 10 13:05:04.937924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Going to run 1 instances... {{(pid=118093) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:05:04.937924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118093) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:05:04.938250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 2/2] 10.222.0.25 () {58 vars in 1244 bytes} [Fri Jun 10 13:05:04 2022] GET /compute/v2.1/os-volumes/abc42eea-e8f2-451c-ac62-968cd4d8c1aa => generated 352 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:05:05.298144 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-35d4de16-9f27-4586-b199-5831338bc88b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.952181 Jun 10 13:05:05.299556 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 1/3] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:05:02 2022] POST /compute/v2.1/servers => generated 380 bytes in 2955 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jun 10 13:05:05.322893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7056db91-9983-4c08-b487-fb869c271a52 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:05.322893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7056db91-9983-4c08-b487-fb869c271a52 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:05.507574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7056db91-9983-4c08-b487-fb869c271a52 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:05.541465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7056db91-9983-4c08-b487-fb869c271a52 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1214 microversion: 2.1 time: 0.234148 Jun 10 13:05:05.544254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 3/4] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:05 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1214 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:05.943280 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cbce5116-fbae-4745-adbe-53dd604ef7f3 tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] Calling method '.version_select at 0x7f93067e93a0>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:06.125605 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cbce5116-fbae-4745-adbe-53dd604ef7f3 tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/abc42eea-e8f2-451c-ac62-968cd4d8c1aa" status: 200 len: 353 microversion: 2.1 time: 0.187544 Jun 10 13:05:06.126266 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 2/5] 10.222.0.25 () {58 vars in 1244 bytes} [Fri Jun 10 13:05:05 2022] GET /compute/v2.1/os-volumes/abc42eea-e8f2-451c-ac62-968cd4d8c1aa => generated 353 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:05:06.137597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-140763c9-e334-4517-b617-e514ed020f2f tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] Calling method '.version_select at 0x7f9307086ee0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:06.253344 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-140763c9-e334-4517-b617-e514ed020f2f tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/abc42eea-e8f2-451c-ac62-968cd4d8c1aa" status: 200 len: 353 microversion: 2.1 time: 0.119372 Jun 10 13:05:06.254379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 4/6] 10.222.0.25 () {58 vars in 1244 bytes} [Fri Jun 10 13:05:06 2022] GET /compute/v2.1/os-volumes/abc42eea-e8f2-451c-ac62-968cd4d8c1aa => generated 353 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:05:06.265730 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1ec5a218-ffcb-4131-a0a9-2a2681fc07e0 tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] Calling method '.version_select at 0x7f9305fc8b80>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:06.562036 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-43a88960-c350-40aa-bde9-b1f1d1bef74c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:06.562794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-43a88960-c350-40aa-bde9-b1f1d1bef74c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:06.610857 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1ec5a218-ffcb-4131-a0a9-2a2681fc07e0 tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/abc42eea-e8f2-451c-ac62-968cd4d8c1aa" status: 202 len: 0 microversion: 2.1 time: 0.345936 Jun 10 13:05:06.610857 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 3/7] 10.222.0.25 () {58 vars in 1247 bytes} [Fri Jun 10 13:05:06 2022] DELETE /compute/v2.1/os-volumes/abc42eea-e8f2-451c-ac62-968cd4d8c1aa => generated 0 bytes in 347 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jun 10 13:05:06.618591 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-bd77d50c-3f9a-4f92-b8da-6a3539b50ebe tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] Calling method '.version_select at 0x7f9305fd7280>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:06.737393 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-bd77d50c-3f9a-4f92-b8da-6a3539b50ebe tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/abc42eea-e8f2-451c-ac62-968cd4d8c1aa" status: 200 len: 352 microversion: 2.1 time: 0.123588 Jun 10 13:05:06.738938 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1244 bytes} [Fri Jun 10 13:05:06 2022] GET /compute/v2.1/os-volumes/abc42eea-e8f2-451c-ac62-968cd4d8c1aa => generated 352 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:05:06.833639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-43a88960-c350-40aa-bde9-b1f1d1bef74c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:06.842162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-43a88960-c350-40aa-bde9-b1f1d1bef74c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1214 microversion: 2.1 time: 0.285165 Jun 10 13:05:06.844901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 5/9] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:06 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1214 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:07.751749 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ac2ee82f-0fb5-4821-a066-f260900d5af2 tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] Calling method '.version_select at 0x7f9305fd7940>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:07.782176 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.wsgi [None req-ac2ee82f-0fb5-4821-a066-f260900d5af2 tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] HTTP exception thrown: Volume abc42eea-e8f2-451c-ac62-968cd4d8c1aa could not be found. Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ac2ee82f-0fb5-4821-a066-f260900d5af2 tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] Request method failure captured: Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: request: GET /compute/v2.1/os-volumes/abc42eea-e8f2-451c-ac62-968cd4d8c1aa HTTP/1.1 Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept: application/json Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept-Encoding: identity Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Connection: close Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Length: 0 Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Type: application/json Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Host: 10.222.0.25 Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Openstack-System-Scope: None Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: User-Agent: python-urllib3/1.26.9 Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Auth-Token: gAAAAABio0F7otW13rPCIH8hKw58lHX-SNX37Qgkl5SrVHuVgc5bwVBF7IikUcR41fn5SgYlxuIeL69792BL1u-y_4C2cPm8_9omo6vgAP97ZThayKYObZv8Q_YV0gwHO51_Ya8vsQFRkJstTDDA1XT4Jyqdo8NkF012KxJlhk0HrIOsRiSLi4w Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Id: None Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Name: None Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Identity-Status: Confirmed Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Is-Admin-Project: True Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Id: default Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Name: Default Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Id: c40c1b8609d249bfb31e3c9bd5cc0623 Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Name: tempest-VolumesGetTestJSON-261545298 Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Role: reader,member Jun 10 13:05:07.786246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Roles: reader,member Jun 10 13:05:07.787349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/c40c1b8609d249bfb31e3c9bd5cc0623"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c40c1b8609d249bfb31e3c9bd5cc0623"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_c40c1b8609d249bfb31e3c9bd5cc0623", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c40c1b8609d249bfb31e3c9bd5cc0623"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:05:07.787349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant: tempest-VolumesGetTestJSON-261545298 Jun 10 13:05:07.787349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Id: c40c1b8609d249bfb31e3c9bd5cc0623 Jun 10 13:05:07.787349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Name: tempest-VolumesGetTestJSON-261545298 Jun 10 13:05:07.787349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User: tempest-VolumesGetTestJSON-261545298-project Jun 10 13:05:07.787349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Id: default Jun 10 13:05:07.787349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Name: Default Jun 10 13:05:07.787349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Id: dad62ca49b6f49c3b6f084accc6bc2f1 Jun 10 13:05:07.787349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Name: tempest-VolumesGetTestJSON-261545298-project Jun 10 13:05:07.787349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: method: Jun 10 13:05:07.787349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: exception: Volume abc42eea-e8f2-451c-ac62-968cd4d8c1aa could not be found. Jun 10 13:05:07.787349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: action_args: {'id': 'abc42eea-e8f2-451c-ac62-968cd4d8c1aa'} Jun 10 13:05:07.787349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi item = cinderclient( Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume abc42eea-e8f2-451c-ac62-968cd4d8c1aa could not be found. (HTTP 404) (Request-ID: req-13bc7512-b300-44a5-9529-a0fa0a8be6a7) Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:05:07.788018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi item = cinderclient( Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jun 10 13:05:07.789084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume abc42eea-e8f2-451c-ac62-968cd4d8c1aa could not be found. Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jun 10 13:05:07.789973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:05:07.790902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume abc42eea-e8f2-451c-ac62-968cd4d8c1aa could not be found. Jun 10 13:05:07.790902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:05:07.790902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:05:07.790902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:05:07.790902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:05:07.790902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:05:07.790902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:05:07.790902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:05:07.790902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:05:07.790902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume abc42eea-e8f2-451c-ac62-968cd4d8c1aa could not be found. Jun 10 13:05:07.790902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:05:07.790902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ac2ee82f-0fb5-4821-a066-f260900d5af2 tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] Returning 404 to user: Volume abc42eea-e8f2-451c-ac62-968cd4d8c1aa could not be found. {{(pid=118093) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:05:07.790902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ac2ee82f-0fb5-4821-a066-f260900d5af2 tempest-VolumesGetTestJSON-261545298 tempest-VolumesGetTestJSON-261545298-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/abc42eea-e8f2-451c-ac62-968cd4d8c1aa" status: 404 len: 109 microversion: 2.1 time: 0.040321 Jun 10 13:05:07.790902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 5/10] 10.222.0.25 () {58 vars in 1244 bytes} [Fri Jun 10 13:05:07 2022] GET /compute/v2.1/os-volumes/abc42eea-e8f2-451c-ac62-968cd4d8c1aa => generated 109 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:05:07.861275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-74aa38bc-18c8-46de-bacd-9f249989afd2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:07.861887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-74aa38bc-18c8-46de-bacd-9f249989afd2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:07.880564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74aa38bc-18c8-46de-bacd-9f249989afd2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:07.881075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74aa38bc-18c8-46de-bacd-9f249989afd2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:07.881852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74aa38bc-18c8-46de-bacd-9f249989afd2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:07.882290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74aa38bc-18c8-46de-bacd-9f249989afd2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:08.166428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-74aa38bc-18c8-46de-bacd-9f249989afd2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:08.181379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-74aa38bc-18c8-46de-bacd-9f249989afd2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1218 microversion: 2.1 time: 0.323729 Jun 10 13:05:08.195202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 6/11] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:07 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1218 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:09.200262 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8773b7a6-ae46-4350-9699-6b1d6c38a796 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:09.201502 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8773b7a6-ae46-4350-9699-6b1d6c38a796 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:09.242910 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8773b7a6-ae46-4350-9699-6b1d6c38a796 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:09.246472 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8773b7a6-ae46-4350-9699-6b1d6c38a796 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:09.246472 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8773b7a6-ae46-4350-9699-6b1d6c38a796 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:09.246472 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8773b7a6-ae46-4350-9699-6b1d6c38a796 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:09.620170 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8773b7a6-ae46-4350-9699-6b1d6c38a796 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:09.645941 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8773b7a6-ae46-4350-9699-6b1d6c38a796 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1266 microversion: 2.1 time: 0.450640 Jun 10 13:05:09.651529 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:09 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1266 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:10.669051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-12e7d565-75c3-4662-8d81-c9a827e049c7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:10.670653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-12e7d565-75c3-4662-8d81-c9a827e049c7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:10.701226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-12e7d565-75c3-4662-8d81-c9a827e049c7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:10.701226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-12e7d565-75c3-4662-8d81-c9a827e049c7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:10.701226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-12e7d565-75c3-4662-8d81-c9a827e049c7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:10.701226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-12e7d565-75c3-4662-8d81-c9a827e049c7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:11.042767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-12e7d565-75c3-4662-8d81-c9a827e049c7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:11.057769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-12e7d565-75c3-4662-8d81-c9a827e049c7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1272 microversion: 2.1 time: 0.387199 Jun 10 13:05:11.064955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:10 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1272 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:12.076511 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1559c5a0-9578-4311-9df9-0ca4f481bbf8 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:12.077811 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1559c5a0-9578-4311-9df9-0ca4f481bbf8 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:12.094346 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1559c5a0-9578-4311-9df9-0ca4f481bbf8 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:12.105548 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1559c5a0-9578-4311-9df9-0ca4f481bbf8 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:12.111456 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1559c5a0-9578-4311-9df9-0ca4f481bbf8 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:12.114340 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1559c5a0-9578-4311-9df9-0ca4f481bbf8 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:13.265770 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1559c5a0-9578-4311-9df9-0ca4f481bbf8 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:13.275594 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1559c5a0-9578-4311-9df9-0ca4f481bbf8 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1272 microversion: 2.1 time: 1.203729 Jun 10 13:05:13.277474 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 7/14] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:12 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1272 bytes in 1206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:13.874247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c2d1e2f3-b87a-4f26-9c5b-ca6a7a924d08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Action: 'create', calling method: .version_select at 0x7f9306fa08b0>, body: {"keypair": {"name": "tempest-keypair-966788633", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL3kc9ph1BunfIVSzzykwngRBGDbcORdOcsyPG9NxXmneEtfmCxPMQsGbugYJD7t8ws0eUSE9iaIF0XOS3kwvPUiAq75vQamVRTq1Ak636wAP3l3Tt4M/L3OQdtepxeVEA=="}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:05:14.014302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-c2d1e2f3-b87a-4f26-9c5b-ca6a7a924d08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Getting quotas for project 28f8244decff453e98e2d372a97a563c. Resources: {'key_pairs'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:05:14.046283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-c2d1e2f3-b87a-4f26-9c5b-ca6a7a924d08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Getting quotas for user b1d1712ea523498f92f258383863b906 and project 28f8244decff453e98e2d372a97a563c. Resources: {'key_pairs'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:05:14.298212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6fbc9147-5a2c-48f6-afb3-bec17960f6d2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:14.301911 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6fbc9147-5a2c-48f6-afb3-bec17960f6d2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:14.315316 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6fbc9147-5a2c-48f6-afb3-bec17960f6d2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:14.320069 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6fbc9147-5a2c-48f6-afb3-bec17960f6d2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:14.322813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6fbc9147-5a2c-48f6-afb3-bec17960f6d2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:14.323535 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6fbc9147-5a2c-48f6-afb3-bec17960f6d2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:14.483239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c2d1e2f3-b87a-4f26-9c5b-ca6a7a924d08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 1.403054 Jun 10 13:05:14.484586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 8/15] 10.222.0.25 () {62 vars in 1199 bytes} [Fri Jun 10 13:05:13 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 1405 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jun 10 13:05:14.884495 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=2 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:14.920646 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=6 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:14.985271 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6fbc9147-5a2c-48f6-afb3-bec17960f6d2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:14.995002 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6fbc9147-5a2c-48f6-afb3-bec17960f6d2 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 0.702062 Jun 10 13:05:15.006466 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:14 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 714 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:16.014185 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b8342cec-53ba-4078-aecf-77a691a00d7c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:16.015626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b8342cec-53ba-4078-aecf-77a691a00d7c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:16.035619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8342cec-53ba-4078-aecf-77a691a00d7c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:16.043278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8342cec-53ba-4078-aecf-77a691a00d7c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:16.048554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8342cec-53ba-4078-aecf-77a691a00d7c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:16.048554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8342cec-53ba-4078-aecf-77a691a00d7c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:16.629745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b8342cec-53ba-4078-aecf-77a691a00d7c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:16.642594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b8342cec-53ba-4078-aecf-77a691a00d7c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 0.633322 Jun 10 13:05:16.644582 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 9/17] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:16 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:17.043024 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-67151863-c7d7-4def-9dc4-126bf9d35d41 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Action: 'create', calling method: .version_select at 0x7f9305e7f160>, body: {"keypair": {"name": "tempest-keypair-1040680043", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNWRrGWupWIpkh1WFDjrAJ3wrTUWyvGPLcKj1bCs1G2HEpMhKeJv5sSzuGFtAmv8hq8PJhTlDU7APNxDGZ18RoJMiZspbxkYTYjLVTYICZKvuF2kywBPiMqJB0+3XUMc/g=="}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:05:17.121658 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-67151863-c7d7-4def-9dc4-126bf9d35d41 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Getting quotas for project 6dace38cecd44f87becbf6dc213851b3. Resources: {'key_pairs'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:05:17.147598 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-67151863-c7d7-4def-9dc4-126bf9d35d41 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Getting quotas for user 9c8b2c27b69a4776ab6b7b7820da02d9 and project 6dace38cecd44f87becbf6dc213851b3. Resources: {'key_pairs'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:05:17.425821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-67151863-c7d7-4def-9dc4-126bf9d35d41 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 1.298164 Jun 10 13:05:17.437068 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 9/18] 10.222.0.25 () {60 vars in 1158 bytes} [Fri Jun 10 13:05:16 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 1308 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:05:17.672670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-79368a23-12ea-4130-9457-b94846df1893 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:17.672670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-79368a23-12ea-4130-9457-b94846df1893 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:17.703065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79368a23-12ea-4130-9457-b94846df1893 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:17.704969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79368a23-12ea-4130-9457-b94846df1893 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:17.707344 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79368a23-12ea-4130-9457-b94846df1893 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:17.709190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79368a23-12ea-4130-9457-b94846df1893 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:18.600133 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a1373449-307b-4f66-9197-bb6d1e9c1d38 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] Calling method '.version_select at 0x7f9305e28ca0>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:18.712086 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a1373449-307b-4f66-9197-bb6d1e9c1d38 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/aa74298c-cd08-4676-9a33-09bed9a48b79" status: 200 len: 350 microversion: 2.1 time: 0.117157 Jun 10 13:05:18.721009 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 10/19] 10.222.0.25 () {58 vars in 1244 bytes} [Fri Jun 10 13:05:18 2022] GET /compute/v2.1/os-volumes/aa74298c-cd08-4676-9a33-09bed9a48b79 => generated 350 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:05:19.141388 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-79368a23-12ea-4130-9457-b94846df1893 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:19.150446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-79368a23-12ea-4130-9457-b94846df1893 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 1.488998 Jun 10 13:05:19.155872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 10/20] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:17 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 1495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:20.177409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-11808dc3-1770-40ee-a016-f5eae5ab3620 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:20.177409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-11808dc3-1770-40ee-a016-f5eae5ab3620 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:20.198858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-11808dc3-1770-40ee-a016-f5eae5ab3620 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:20.201835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-11808dc3-1770-40ee-a016-f5eae5ab3620 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:20.206710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-11808dc3-1770-40ee-a016-f5eae5ab3620 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:20.210070 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-11808dc3-1770-40ee-a016-f5eae5ab3620 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:20.353474 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-e3034ad5-5c12-46e9-b24f-5c6d26fb195a req-deeeb6e0-5070-4e2e-b7ef-fddb351b03b1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1", "tag": "9e2b9907-e960-46c3-ac8f-00608c9e9b15"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:05:20.380870 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-e3034ad5-5c12-46e9-b24f-5c6d26fb195a req-deeeb6e0-5070-4e2e-b7ef-fddb351b03b1 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:20.383171 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-e3034ad5-5c12-46e9-b24f-5c6d26fb195a req-deeeb6e0-5070-4e2e-b7ef-fddb351b03b1 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:20.460899 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-e3034ad5-5c12-46e9-b24f-5c6d26fb195a req-deeeb6e0-5070-4e2e-b7ef-fddb351b03b1 service nova] Creating event network-changed:9e2b9907-e960-46c3-ac8f-00608c9e9b15 for instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 on hv-cinder-maste Jun 10 13:05:20.461568 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-e3034ad5-5c12-46e9-b24f-5c6d26fb195a req-deeeb6e0-5070-4e2e-b7ef-fddb351b03b1 service nova] Acquired lock "compute-rpcapi-router" {{(pid=118093) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jun 10 13:05:20.465934 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-82e13b82-96d6-431a-b428-ff194234564f None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:20.475823 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-82e13b82-96d6-431a-b428-ff194234564f None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:20.475823 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-82e13b82-96d6-431a-b428-ff194234564f None None] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:20.475823 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-82e13b82-96d6-431a-b428-ff194234564f None None] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:20.496584 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.compute.rpcapi [None req-82e13b82-96d6-431a-b428-ff194234564f None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jun 10 13:05:20.498161 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-82e13b82-96d6-431a-b428-ff194234564f None None] Releasing lock "compute-rpcapi-router" {{(pid=118093) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jun 10 13:05:20.555777 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-82e13b82-96d6-431a-b428-ff194234564f None None] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.782546 Jun 10 13:05:20.557726 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 11/21] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:05:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 787 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:05:20.891452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-11808dc3-1770-40ee-a016-f5eae5ab3620 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:20.916809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-11808dc3-1770-40ee-a016-f5eae5ab3620 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 0.734378 Jun 10 13:05:20.916809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 11/22] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:20 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 736 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:21.266109 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-77178e37-3692-491c-a19d-7d0d474e7ec3 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] Calling method '.version_select at 0x7f9305e908b0>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:21.395785 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-77178e37-3692-491c-a19d-7d0d474e7ec3 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/c178de2c-0d8e-45ab-9676-674618b30ca3" status: 200 len: 351 microversion: 2.1 time: 0.134316 Jun 10 13:05:21.396429 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 12/23] 10.222.0.25 () {58 vars in 1244 bytes} [Fri Jun 10 13:05:21 2022] GET /compute/v2.1/os-volumes/c178de2c-0d8e-45ab-9676-674618b30ca3 => generated 351 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:05:21.935375 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-354a2b04-3804-4235-bcdf-7071a143fb46 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:21.937212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-354a2b04-3804-4235-bcdf-7071a143fb46 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:21.973362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-354a2b04-3804-4235-bcdf-7071a143fb46 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:22.007716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-354a2b04-3804-4235-bcdf-7071a143fb46 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:22.007716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-354a2b04-3804-4235-bcdf-7071a143fb46 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:22.007716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-354a2b04-3804-4235-bcdf-7071a143fb46 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:22.477648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-354a2b04-3804-4235-bcdf-7071a143fb46 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:22.489817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-354a2b04-3804-4235-bcdf-7071a143fb46 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 0.561842 Jun 10 13:05:22.492439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 12/24] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:21 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 565 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:22.728729 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1716413195", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a83e307f-ca79-4dcf-bb7e-cc270b1f93af"}], "security_groups": [{"name": "tempest-securitygroup--887280941"}], "key_name": "tempest-keypair-966788633", "user_data": "}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:05:22.942109 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.network.neutron [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] validate_networks() for [('a83e307f-ca79-4dcf-bb7e-cc270b1f93af', None, None, None, None, None)] {{(pid=118093) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:05:23.511690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3fd13dec-cd75-45df-bebf-33af6a7f8a84 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:23.515456 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3fd13dec-cd75-45df-bebf-33af6a7f8a84 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:23.537436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3fd13dec-cd75-45df-bebf-33af6a7f8a84 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:23.537436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3fd13dec-cd75-45df-bebf-33af6a7f8a84 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:23.537436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3fd13dec-cd75-45df-bebf-33af6a7f8a84 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:23.537436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3fd13dec-cd75-45df-bebf-33af6a7f8a84 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:23.787548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3fd13dec-cd75-45df-bebf-33af6a7f8a84 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:23.796894 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3fd13dec-cd75-45df-bebf-33af6a7f8a84 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 0.289236 Jun 10 13:05:23.798147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 13/25] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:23 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:05:23.852930 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Flavor limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:05:23.853436 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Image limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:05:23.853832 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Flavor pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:05:23.854300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Image pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:05:23.854718 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:05:23.868073 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:23.868905 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:23.869738 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:23.870318 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:23.890754 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Getting quotas for project 28f8244decff453e98e2d372a97a563c. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:05:23.906239 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Getting quotas for user b1d1712ea523498f92f258383863b906 and project 28f8244decff453e98e2d372a97a563c. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:05:23.920043 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Going to run 1 instances... {{(pid=118093) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:05:23.922836 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118093) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:05:24.214080 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-62fbc68d-f8a5-4a44-aabf-f931c0a006e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 1.490005 Jun 10 13:05:24.223214 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 13/26] 10.222.0.25 () {62 vars in 1187 bytes} [Fri Jun 10 13:05:22 2022] POST /compute/v2.1/servers => generated 401 bytes in 1500 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jun 10 13:05:24.226179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3b4b7706-e698-4079-ad0f-3b4e2d8fc347 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:24.230540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3b4b7706-e698-4079-ad0f-3b4e2d8fc347 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:24.244037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=8 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:24.438404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3b4b7706-e698-4079-ad0f-3b4e2d8fc347 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:24.457116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3b4b7706-e698-4079-ad0f-3b4e2d8fc347 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:24.459635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3b4b7706-e698-4079-ad0f-3b4e2d8fc347 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:24.461144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3b4b7706-e698-4079-ad0f-3b4e2d8fc347 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1281 microversion: 2.20 time: 0.239376 Jun 10 13:05:24.463023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 14/27] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:24 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1281 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:05:24.814493 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4b3fcd90-a05f-4b23-87cc-c84245bf2582 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] Calling method '.version_select at 0x7f9305e39d30>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:24.913309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dc756dec-7598-40e9-9c6b-54da79cfffc0 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:24.916393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dc756dec-7598-40e9-9c6b-54da79cfffc0 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:24.934171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc756dec-7598-40e9-9c6b-54da79cfffc0 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:24.935702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc756dec-7598-40e9-9c6b-54da79cfffc0 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:24.937329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc756dec-7598-40e9-9c6b-54da79cfffc0 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:24.938506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc756dec-7598-40e9-9c6b-54da79cfffc0 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:25.249363 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4b3fcd90-a05f-4b23-87cc-c84245bf2582 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/9735e084-6207-4adc-ad79-d495a2adfa96" status: 200 len: 351 microversion: 2.1 time: 0.445992 Jun 10 13:05:25.263701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 14/28] 10.222.0.25 () {58 vars in 1244 bytes} [Fri Jun 10 13:05:24 2022] GET /compute/v2.1/os-volumes/9735e084-6207-4adc-ad79-d495a2adfa96 => generated 351 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:05:25.279322 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e3d10344-22bf-4fb2-a0e7-fc17d3361c2f tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] Calling method '.version_select at 0x7f9305e905e0>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:25.282677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dc756dec-7598-40e9-9c6b-54da79cfffc0 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:25.291620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dc756dec-7598-40e9-9c6b-54da79cfffc0 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 0.385893 Jun 10 13:05:25.292188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 15/29] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:24 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:05:25.483275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-43bffd24-2037-4fd9-99f9-cd8b033b01c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:25.490773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-43bffd24-2037-4fd9-99f9-cd8b033b01c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:25.549027 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e3d10344-22bf-4fb2-a0e7-fc17d3361c2f tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] 10.222.0.25 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.280193 Jun 10 13:05:25.549027 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 15/30] 10.222.0.25 () {58 vars in 1133 bytes} [Fri Jun 10 13:05:25 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:25.564744 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3a9e72eb-d274-4d5d-9a57-5898f5c1ffef tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] Calling method '.version_select at 0x7f9305e90280>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:25.605836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-43bffd24-2037-4fd9-99f9-cd8b033b01c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:25.644170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-43bffd24-2037-4fd9-99f9-cd8b033b01c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:25.648338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-43bffd24-2037-4fd9-99f9-cd8b033b01c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:25.650931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-43bffd24-2037-4fd9-99f9-cd8b033b01c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1281 microversion: 2.20 time: 0.174137 Jun 10 13:05:25.652446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 16/31] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:25 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1281 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:25.692908 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3a9e72eb-d274-4d5d-9a57-5898f5c1ffef tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] 10.222.0.25 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.133366 Jun 10 13:05:25.694071 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 16/32] 10.222.0.25 () {58 vars in 1148 bytes} [Fri Jun 10 13:05:25 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 695 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:05:25.706590 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9c563793-bef8-4a1d-9bb9-a66a7f6087e4 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] Calling method '.version_select at 0x7f93066eb4c0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:25.766410 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1625403030", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a63fb262-f8e3-4ba4-b86b-b0f4cec54e28"}], "security_groups": [{"name": "tempest-securitygroup--261225858"}], "key_name": "tempest-keypair-1040680043", "user_data": "}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:05:25.818456 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9c563793-bef8-4a1d-9bb9-a66a7f6087e4 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] 10.222.0.25 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.115731 Jun 10 13:05:25.819081 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 17/33] 10.222.0.25 () {58 vars in 1133 bytes} [Fri Jun 10 13:05:25 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:05:25.830330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5dda28a9-061b-469e-86e3-a634ba210fd7 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] Calling method '.version_select at 0x7f93066e34c0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:25.880982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.network.neutron [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] validate_networks() for [('a63fb262-f8e3-4ba4-b86b-b0f4cec54e28', None, None, None, None, None)] {{(pid=118093) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:05:25.915775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5dda28a9-061b-469e-86e3-a634ba210fd7 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] 10.222.0.25 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.088996 Jun 10 13:05:25.916388 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 18/34] 10.222.0.25 () {58 vars in 1166 bytes} [Fri Jun 10 13:05:25 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:05:25.927638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-599aad90-d352-493d-88ca-5f405ed557b8 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] Calling method '.version_select at 0x7f93066f8ca0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:26.010965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-599aad90-d352-493d-88ca-5f405ed557b8 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.086932 Jun 10 13:05:26.012630 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 19/35] 10.222.0.25 () {58 vars in 1169 bytes} [Fri Jun 10 13:05:25 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 695 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:05:26.025554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3c35c9aa-ba4c-425e-a266-6d7c9824f726 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] Calling method '.version_select at 0x7f93066ebca0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:26.125934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3c35c9aa-ba4c-425e-a266-6d7c9824f726 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.104285 Jun 10 13:05:26.126759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 20/36] 10.222.0.25 () {58 vars in 1154 bytes} [Fri Jun 10 13:05:26 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:05:26.147507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a949aa3a-a552-4eaa-a55a-71697480dc30 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] Calling method '.version_select at 0x7f9306f35040>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:26.266216 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a949aa3a-a552-4eaa-a55a-71697480dc30 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.123871 Jun 10 13:05:26.266858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 21/37] 10.222.0.25 () {58 vars in 1187 bytes} [Fri Jun 10 13:05:26 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:05:26.278409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0fd1ec5d-afc5-4884-8a5a-db5f8033f5d9 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] Calling method '.version_select at 0x7f9306f35160>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:26.442307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Flavor limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:05:26.442775 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Image limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:05:26.443193 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Flavor pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:05:26.443599 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Image pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:05:26.444041 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:05:26.453048 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:26.453498 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:26.454331 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:26.454788 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:26.471572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Getting quotas for project 6dace38cecd44f87becbf6dc213851b3. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:05:26.475506 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Getting quotas for user 9c8b2c27b69a4776ab6b7b7820da02d9 and project 6dace38cecd44f87becbf6dc213851b3. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:05:26.487215 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Going to run 1 instances... {{(pid=118093) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:05:26.489642 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118093) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:05:26.613715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d428866b-1375-4e07-8bc2-1ebc9b75e323 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.854089 Jun 10 13:05:26.615057 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 17/38] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:05:25 2022] POST /compute/v2.1/servers => generated 401 bytes in 856 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jun 10 13:05:26.620604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f45c7144-a308-4d74-96e9-b272db14cfd3 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:26.622143 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f45c7144-a308-4d74-96e9-b272db14cfd3 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:26.630830 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f45c7144-a308-4d74-96e9-b272db14cfd3 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:26.631374 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f45c7144-a308-4d74-96e9-b272db14cfd3 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:26.632462 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f45c7144-a308-4d74-96e9-b272db14cfd3 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:26.633039 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f45c7144-a308-4d74-96e9-b272db14cfd3 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:26.677309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0fd1ec5d-afc5-4884-8a5a-db5f8033f5d9 tempest-VolumesTestJSON-1231281922 tempest-VolumesTestJSON-1231281922-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.399647 Jun 10 13:05:26.677915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 22/39] 10.222.0.25 () {58 vars in 1154 bytes} [Fri Jun 10 13:05:26 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:26.697703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-71614771-7023-423b-ab27-405508613814 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:26.698329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-71614771-7023-423b-ab27-405508613814 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:26.793300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-71614771-7023-423b-ab27-405508613814 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:26.801523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-71614771-7023-423b-ab27-405508613814 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1238 microversion: 2.1 time: 0.120837 Jun 10 13:05:26.802084 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 23/40] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:26 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1238 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:26.821045 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-026657f4-76ec-4b26-864d-8016154fad1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:26.821914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-026657f4-76ec-4b26-864d-8016154fad1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:26.829978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-026657f4-76ec-4b26-864d-8016154fad1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:26.830620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-026657f4-76ec-4b26-864d-8016154fad1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:26.831546 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-026657f4-76ec-4b26-864d-8016154fad1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:26.832119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-026657f4-76ec-4b26-864d-8016154fad1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:26.883746 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f45c7144-a308-4d74-96e9-b272db14cfd3 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:26.894227 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f45c7144-a308-4d74-96e9-b272db14cfd3 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 0.277679 Jun 10 13:05:26.894851 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 18/41] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:26 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:26.921185 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-026657f4-76ec-4b26-864d-8016154fad1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:26.929160 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-026657f4-76ec-4b26-864d-8016154fad1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:26.930774 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-026657f4-76ec-4b26-864d-8016154fad1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:26.931876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-026657f4-76ec-4b26-864d-8016154fad1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1285 microversion: 2.20 time: 0.127899 Jun 10 13:05:26.932429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 24/42] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:26 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1285 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:27.828880 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9e01592d-8afb-4e5d-938c-5be3e1ed2a50 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:27.829505 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9e01592d-8afb-4e5d-938c-5be3e1ed2a50 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:27.886460 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9e01592d-8afb-4e5d-938c-5be3e1ed2a50 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:27.901413 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9e01592d-8afb-4e5d-938c-5be3e1ed2a50 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1238 microversion: 2.1 time: 0.072506 Jun 10 13:05:27.901413 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 19/43] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:27 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1238 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:27.911214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e3513159-1a58-489a-871e-608c0b001aa7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:27.911810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e3513159-1a58-489a-871e-608c0b001aa7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:27.918592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e3513159-1a58-489a-871e-608c0b001aa7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:27.919444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e3513159-1a58-489a-871e-608c0b001aa7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:27.920685 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e3513159-1a58-489a-871e-608c0b001aa7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:27.921544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e3513159-1a58-489a-871e-608c0b001aa7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:27.952259 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3af4b2df-50c2-4b53-8d66-365339b19bfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:27.953092 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3af4b2df-50c2-4b53-8d66-365339b19bfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:27.961004 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3af4b2df-50c2-4b53-8d66-365339b19bfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:27.961514 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3af4b2df-50c2-4b53-8d66-365339b19bfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:27.962351 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3af4b2df-50c2-4b53-8d66-365339b19bfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:27.962863 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3af4b2df-50c2-4b53-8d66-365339b19bfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:28.028872 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3af4b2df-50c2-4b53-8d66-365339b19bfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:28.036983 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3af4b2df-50c2-4b53-8d66-365339b19bfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:28.038487 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3af4b2df-50c2-4b53-8d66-365339b19bfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:28.039591 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3af4b2df-50c2-4b53-8d66-365339b19bfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1285 microversion: 2.20 time: 0.091478 Jun 10 13:05:28.040250 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 20/44] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:27 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1285 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:28.074106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e3513159-1a58-489a-871e-608c0b001aa7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:28.081875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e3513159-1a58-489a-871e-608c0b001aa7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 0.174345 Jun 10 13:05:28.082470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 25/45] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:27 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:28.914652 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6c6f5fed-bab9-4887-8821-789f26d5b547 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:28.919537 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6c6f5fed-bab9-4887-8821-789f26d5b547 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:28.921799 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c6f5fed-bab9-4887-8821-789f26d5b547 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:28.922243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c6f5fed-bab9-4887-8821-789f26d5b547 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:28.922992 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c6f5fed-bab9-4887-8821-789f26d5b547 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:28.923421 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c6f5fed-bab9-4887-8821-789f26d5b547 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:28.983670 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6c6f5fed-bab9-4887-8821-789f26d5b547 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:28.991257 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6c6f5fed-bab9-4887-8821-789f26d5b547 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1242 microversion: 2.1 time: 0.080379 Jun 10 13:05:28.991812 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 21/46] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:28 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1242 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:29.057559 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5e03fa8c-d607-439c-a88a-daf444cfb2f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:29.058308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5e03fa8c-d607-439c-a88a-daf444cfb2f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:29.064494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e03fa8c-d607-439c-a88a-daf444cfb2f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:29.064991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e03fa8c-d607-439c-a88a-daf444cfb2f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:29.066033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e03fa8c-d607-439c-a88a-daf444cfb2f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:29.066474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e03fa8c-d607-439c-a88a-daf444cfb2f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:29.107999 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-655ca76a-8dbf-4936-b7f3-d5c489f866fc tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:29.107999 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-655ca76a-8dbf-4936-b7f3-d5c489f866fc tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:29.111843 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-655ca76a-8dbf-4936-b7f3-d5c489f866fc tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:29.112687 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-655ca76a-8dbf-4936-b7f3-d5c489f866fc tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:29.113494 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-655ca76a-8dbf-4936-b7f3-d5c489f866fc tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:29.114462 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-655ca76a-8dbf-4936-b7f3-d5c489f866fc tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:29.130299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5e03fa8c-d607-439c-a88a-daf444cfb2f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:29.138593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5e03fa8c-d607-439c-a88a-daf444cfb2f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:29.140133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5e03fa8c-d607-439c-a88a-daf444cfb2f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:29.141337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5e03fa8c-d607-439c-a88a-daf444cfb2f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1285 microversion: 2.20 time: 0.087627 Jun 10 13:05:29.141827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 26/47] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:29 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1285 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:29.255754 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-655ca76a-8dbf-4936-b7f3-d5c489f866fc tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:29.264021 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-655ca76a-8dbf-4936-b7f3-d5c489f866fc tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 0.169258 Jun 10 13:05:29.264266 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 22/48] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:29 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:05:30.007658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c6490465-6863-403f-bfd5-a6a8bc587bac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:30.008268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c6490465-6863-403f-bfd5-a6a8bc587bac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:30.010136 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=54,nova_api:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:30.014954 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6490465-6863-403f-bfd5-a6a8bc587bac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:30.015431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6490465-6863-403f-bfd5-a6a8bc587bac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:30.016191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6490465-6863-403f-bfd5-a6a8bc587bac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:30.016621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6490465-6863-403f-bfd5-a6a8bc587bac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:30.075937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c6490465-6863-403f-bfd5-a6a8bc587bac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:30.083277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c6490465-6863-403f-bfd5-a6a8bc587bac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1242 microversion: 2.1 time: 0.079825 Jun 10 13:05:30.083832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 27/49] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:30 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1242 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:30.159999 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0dc00372-a691-43cb-85cf-859f194cc14e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:30.160607 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0dc00372-a691-43cb-85cf-859f194cc14e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:30.166665 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0dc00372-a691-43cb-85cf-859f194cc14e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:30.167114 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0dc00372-a691-43cb-85cf-859f194cc14e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:30.168364 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0dc00372-a691-43cb-85cf-859f194cc14e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:30.168838 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0dc00372-a691-43cb-85cf-859f194cc14e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:30.248067 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0dc00372-a691-43cb-85cf-859f194cc14e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:30.256044 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0dc00372-a691-43cb-85cf-859f194cc14e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:30.257584 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0dc00372-a691-43cb-85cf-859f194cc14e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:30.258881 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0dc00372-a691-43cb-85cf-859f194cc14e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1285 microversion: 2.20 time: 0.102880 Jun 10 13:05:30.259598 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 23/50] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:30 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1285 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:30.280814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0cf23660-6600-4d5c-9bc2-4a99b60238e9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:30.281510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0cf23660-6600-4d5c-9bc2-4a99b60238e9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:30.288272 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0cf23660-6600-4d5c-9bc2-4a99b60238e9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:30.288847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0cf23660-6600-4d5c-9bc2-4a99b60238e9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:30.289847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0cf23660-6600-4d5c-9bc2-4a99b60238e9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:30.290412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0cf23660-6600-4d5c-9bc2-4a99b60238e9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:30.295623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=73 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:30.456888 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0cf23660-6600-4d5c-9bc2-4a99b60238e9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:30.464363 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0cf23660-6600-4d5c-9bc2-4a99b60238e9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 0.187825 Jun 10 13:05:30.465537 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 28/51] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:30 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:31.100139 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6c1a36bd-169d-4e7f-8b6b-434aa8f4aa14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:31.101187 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6c1a36bd-169d-4e7f-8b6b-434aa8f4aa14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:31.108579 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c1a36bd-169d-4e7f-8b6b-434aa8f4aa14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:31.112839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c1a36bd-169d-4e7f-8b6b-434aa8f4aa14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:31.114364 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c1a36bd-169d-4e7f-8b6b-434aa8f4aa14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:31.115246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c1a36bd-169d-4e7f-8b6b-434aa8f4aa14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:31.223460 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6c1a36bd-169d-4e7f-8b6b-434aa8f4aa14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:31.251319 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6c1a36bd-169d-4e7f-8b6b-434aa8f4aa14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1290 microversion: 2.1 time: 0.142992 Jun 10 13:05:31.277043 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 24/52] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:31 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1290 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:31.278924 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-87063487-8baa-469f-93ad-0a0a90d40da7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:31.285508 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-87063487-8baa-469f-93ad-0a0a90d40da7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:31.288480 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87063487-8baa-469f-93ad-0a0a90d40da7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:31.292884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87063487-8baa-469f-93ad-0a0a90d40da7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:31.295197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87063487-8baa-469f-93ad-0a0a90d40da7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:31.295736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87063487-8baa-469f-93ad-0a0a90d40da7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:31.393671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-87063487-8baa-469f-93ad-0a0a90d40da7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:31.402038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-87063487-8baa-469f-93ad-0a0a90d40da7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:31.403577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-87063487-8baa-469f-93ad-0a0a90d40da7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:31.404744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-87063487-8baa-469f-93ad-0a0a90d40da7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1333 microversion: 2.20 time: 0.130921 Jun 10 13:05:31.415397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 29/53] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:31 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1333 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:05:31.485568 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a3cd83ad-c556-4d3e-9844-b5815e09f2b9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:31.486321 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a3cd83ad-c556-4d3e-9844-b5815e09f2b9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:31.493001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3cd83ad-c556-4d3e-9844-b5815e09f2b9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:31.493618 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3cd83ad-c556-4d3e-9844-b5815e09f2b9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:31.494478 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3cd83ad-c556-4d3e-9844-b5815e09f2b9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:31.495051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3cd83ad-c556-4d3e-9844-b5815e09f2b9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:31.724358 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a3cd83ad-c556-4d3e-9844-b5815e09f2b9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:31.732853 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a3cd83ad-c556-4d3e-9844-b5815e09f2b9 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 0.251256 Jun 10 13:05:31.733553 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 25/54] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:31 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:32.267691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dc5b14a4-3e9c-414d-a5bb-6684e26d85f1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:32.268236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dc5b14a4-3e9c-414d-a5bb-6684e26d85f1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:32.275555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc5b14a4-3e9c-414d-a5bb-6684e26d85f1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:32.276005 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc5b14a4-3e9c-414d-a5bb-6684e26d85f1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:32.276797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc5b14a4-3e9c-414d-a5bb-6684e26d85f1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:32.277809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc5b14a4-3e9c-414d-a5bb-6684e26d85f1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:32.366665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dc5b14a4-3e9c-414d-a5bb-6684e26d85f1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:32.385144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dc5b14a4-3e9c-414d-a5bb-6684e26d85f1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1308 microversion: 2.1 time: 0.121353 Jun 10 13:05:32.385718 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 30/55] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:32 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1308 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:32.425080 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-506137c5-587d-492d-ae8f-7c05bb56ff5e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:32.425702 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-506137c5-587d-492d-ae8f-7c05bb56ff5e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:32.435262 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-506137c5-587d-492d-ae8f-7c05bb56ff5e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:32.437060 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-506137c5-587d-492d-ae8f-7c05bb56ff5e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:32.437854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-506137c5-587d-492d-ae8f-7c05bb56ff5e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:32.438693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-506137c5-587d-492d-ae8f-7c05bb56ff5e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:32.601272 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-506137c5-587d-492d-ae8f-7c05bb56ff5e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:32.611896 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-506137c5-587d-492d-ae8f-7c05bb56ff5e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:32.613387 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-506137c5-587d-492d-ae8f-7c05bb56ff5e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:32.614787 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-506137c5-587d-492d-ae8f-7c05bb56ff5e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1341 microversion: 2.20 time: 0.195311 Jun 10 13:05:32.625850 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 26/56] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:32 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1341 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:32.751748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5563ef8f-2dc4-46a7-b3c5-3a358f2f5276 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:32.752396 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5563ef8f-2dc4-46a7-b3c5-3a358f2f5276 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:32.758540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5563ef8f-2dc4-46a7-b3c5-3a358f2f5276 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:32.760831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5563ef8f-2dc4-46a7-b3c5-3a358f2f5276 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:32.760831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5563ef8f-2dc4-46a7-b3c5-3a358f2f5276 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:32.760831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5563ef8f-2dc4-46a7-b3c5-3a358f2f5276 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:32.902475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5563ef8f-2dc4-46a7-b3c5-3a358f2f5276 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:32.919032 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5563ef8f-2dc4-46a7-b3c5-3a358f2f5276 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 0.170445 Jun 10 13:05:32.931425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 31/57] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:32 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:33.156667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-747f22c4-f11c-4cd3-bd2c-b8f4772c1c5f req-d5ade718-5afb-468f-8578-c10a2bcfe32f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1", "name": "network-vif-plugged", "status": "completed", "tag": "9e2b9907-e960-46c3-ac8f-00608c9e9b15"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:05:33.165016 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-747f22c4-f11c-4cd3-bd2c-b8f4772c1c5f req-d5ade718-5afb-468f-8578-c10a2bcfe32f service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:33.165460 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-747f22c4-f11c-4cd3-bd2c-b8f4772c1c5f req-d5ade718-5afb-468f-8578-c10a2bcfe32f service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:33.180320 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-747f22c4-f11c-4cd3-bd2c-b8f4772c1c5f req-d5ade718-5afb-468f-8578-c10a2bcfe32f service nova] Creating event network-vif-plugged:9e2b9907-e960-46c3-ac8f-00608c9e9b15 for instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 on hv-cinder-maste Jun 10 13:05:33.187300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-747f22c4-f11c-4cd3-bd2c-b8f4772c1c5f req-d5ade718-5afb-468f-8578-c10a2bcfe32f service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.088830 Jun 10 13:05:33.188651 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 27/58] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:05:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:05:33.404744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b41ba147-f2a1-463b-84e1-b72289660035 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:33.408601 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b41ba147-f2a1-463b-84e1-b72289660035 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:33.413257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b41ba147-f2a1-463b-84e1-b72289660035 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:33.413827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b41ba147-f2a1-463b-84e1-b72289660035 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:33.415200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b41ba147-f2a1-463b-84e1-b72289660035 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:33.415766 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b41ba147-f2a1-463b-84e1-b72289660035 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:33.646307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ad76f4f2-3d91-43fa-b707-cfd28c42ed5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:33.648510 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ad76f4f2-3d91-43fa-b707-cfd28c42ed5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:33.656283 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=10 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:33.658670 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ad76f4f2-3d91-43fa-b707-cfd28c42ed5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:33.658896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b41ba147-f2a1-463b-84e1-b72289660035 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:33.659590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ad76f4f2-3d91-43fa-b707-cfd28c42ed5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:33.660593 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ad76f4f2-3d91-43fa-b707-cfd28c42ed5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:33.661412 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ad76f4f2-3d91-43fa-b707-cfd28c42ed5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:33.667281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b41ba147-f2a1-463b-84e1-b72289660035 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.268524 Jun 10 13:05:33.668962 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=79 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:33.681107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 32/59] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:33 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:33.832470 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ad76f4f2-3d91-43fa-b707-cfd28c42ed5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:33.845878 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ad76f4f2-3d91-43fa-b707-cfd28c42ed5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:33.848157 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ad76f4f2-3d91-43fa-b707-cfd28c42ed5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:33.850238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ad76f4f2-3d91-43fa-b707-cfd28c42ed5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1339 microversion: 2.20 time: 0.211744 Jun 10 13:05:33.851972 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 28/60] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:33 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1339 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:05:33.937040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0fc9b12e-84bc-4c27-a493-a7c0ec3c16a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:33.937657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0fc9b12e-84bc-4c27-a493-a7c0ec3c16a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:33.944913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fc9b12e-84bc-4c27-a493-a7c0ec3c16a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:33.948718 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fc9b12e-84bc-4c27-a493-a7c0ec3c16a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:33.948718 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fc9b12e-84bc-4c27-a493-a7c0ec3c16a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:33.948718 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fc9b12e-84bc-4c27-a493-a7c0ec3c16a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:34.126514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0fc9b12e-84bc-4c27-a493-a7c0ec3c16a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:34.133353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0fc9b12e-84bc-4c27-a493-a7c0ec3c16a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 0.200375 Jun 10 13:05:34.143844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 33/61] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:33 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:34.689961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-73236c60-4fa1-4405-a62a-19f48715175e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:34.689961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-73236c60-4fa1-4405-a62a-19f48715175e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:34.696141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-73236c60-4fa1-4405-a62a-19f48715175e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:34.696728 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-73236c60-4fa1-4405-a62a-19f48715175e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:34.698003 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-73236c60-4fa1-4405-a62a-19f48715175e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:34.698512 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-73236c60-4fa1-4405-a62a-19f48715175e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:34.874276 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bae023d6-de0f-48f9-aac6-472eb68edb8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:34.875283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bae023d6-de0f-48f9-aac6-472eb68edb8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:34.887612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bae023d6-de0f-48f9-aac6-472eb68edb8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:34.888055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bae023d6-de0f-48f9-aac6-472eb68edb8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:34.890219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bae023d6-de0f-48f9-aac6-472eb68edb8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:34.895280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bae023d6-de0f-48f9-aac6-472eb68edb8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:35.104916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bae023d6-de0f-48f9-aac6-472eb68edb8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:35.117256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bae023d6-de0f-48f9-aac6-472eb68edb8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:35.117256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bae023d6-de0f-48f9-aac6-472eb68edb8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:35.126897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bae023d6-de0f-48f9-aac6-472eb68edb8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.258949 Jun 10 13:05:35.140355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 34/62] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:34 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:35.168494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-1403f576-c5c7-4d2c-9541-039dc004c8e5 req-6f91e45a-1d3c-434a-8285-0a008f4e2d69 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9e403cf8-5aee-49f3-bbbc-600263bd148a", "tag": "9615dc48-0838-44bb-9298-f3df821b26c7"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:05:35.168494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-1403f576-c5c7-4d2c-9541-039dc004c8e5 req-6f91e45a-1d3c-434a-8285-0a008f4e2d69 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:35.168494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-1403f576-c5c7-4d2c-9541-039dc004c8e5 req-6f91e45a-1d3c-434a-8285-0a008f4e2d69 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:35.183244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-1403f576-c5c7-4d2c-9541-039dc004c8e5 req-6f91e45a-1d3c-434a-8285-0a008f4e2d69 service nova] Creating event network-changed:9615dc48-0838-44bb-9298-f3df821b26c7 for instance 9e403cf8-5aee-49f3-bbbc-600263bd148a on hv-cinder-maste Jun 10 13:05:35.183843 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-1403f576-c5c7-4d2c-9541-039dc004c8e5 req-6f91e45a-1d3c-434a-8285-0a008f4e2d69 service nova] Acquired lock "compute-rpcapi-router" {{(pid=118092) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jun 10 13:05:35.185157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-549bd4d6-0992-4a84-8398-7c28b9b07894 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:35.185639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-549bd4d6-0992-4a84-8398-7c28b9b07894 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:35.186523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-549bd4d6-0992-4a84-8398-7c28b9b07894 None None] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:35.186955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-549bd4d6-0992-4a84-8398-7c28b9b07894 None None] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:35.203608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.compute.rpcapi [None req-549bd4d6-0992-4a84-8398-7c28b9b07894 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jun 10 13:05:35.203608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-549bd4d6-0992-4a84-8398-7c28b9b07894 None None] Releasing lock "compute-rpcapi-router" {{(pid=118092) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jun 10 13:05:35.215478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-549bd4d6-0992-4a84-8398-7c28b9b07894 None None] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072657 Jun 10 13:05:35.216341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 35/63] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:05:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:05:35.222356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fb247b9b-2b56-46ea-ae02-716ee5033706 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:35.223072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fb247b9b-2b56-46ea-ae02-716ee5033706 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:35.231528 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb247b9b-2b56-46ea-ae02-716ee5033706 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:35.232263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb247b9b-2b56-46ea-ae02-716ee5033706 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:35.233176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb247b9b-2b56-46ea-ae02-716ee5033706 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:35.233686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb247b9b-2b56-46ea-ae02-716ee5033706 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:35.438762 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-73236c60-4fa1-4405-a62a-19f48715175e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:35.446158 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-73236c60-4fa1-4405-a62a-19f48715175e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.764425 Jun 10 13:05:35.456334 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 29/64] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:34 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 776 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:05:35.462446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fb247b9b-2b56-46ea-ae02-716ee5033706 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:35.470484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fb247b9b-2b56-46ea-ae02-716ee5033706 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1314 microversion: 2.1 time: 0.252893 Jun 10 13:05:35.471059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 36/65] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:35 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1314 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:36.149416 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d843c5dc-5d40-436c-b863-40353a2404d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:36.150130 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d843c5dc-5d40-436c-b863-40353a2404d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:36.164827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d843c5dc-5d40-436c-b863-40353a2404d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:36.165268 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d843c5dc-5d40-436c-b863-40353a2404d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:36.166053 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d843c5dc-5d40-436c-b863-40353a2404d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:36.166689 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d843c5dc-5d40-436c-b863-40353a2404d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:36.405285 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d843c5dc-5d40-436c-b863-40353a2404d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:36.413719 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d843c5dc-5d40-436c-b863-40353a2404d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:36.422667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d843c5dc-5d40-436c-b863-40353a2404d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:36.431819 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d843c5dc-5d40-436c-b863-40353a2404d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.288036 Jun 10 13:05:36.432404 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 30/66] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:36 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:36.464719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5520ba0b-4bd5-4043-a0c5-999f48cc1dbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:36.465355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5520ba0b-4bd5-4043-a0c5-999f48cc1dbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:36.476698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5520ba0b-4bd5-4043-a0c5-999f48cc1dbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:36.477344 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5520ba0b-4bd5-4043-a0c5-999f48cc1dbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:36.481033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5520ba0b-4bd5-4043-a0c5-999f48cc1dbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:36.483027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5520ba0b-4bd5-4043-a0c5-999f48cc1dbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:36.510367 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ab24a67e-2d33-4f72-a4ea-d6601a4e5e57 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:36.510367 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ab24a67e-2d33-4f72-a4ea-d6601a4e5e57 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:36.510367 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=6 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:36.516204 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ab24a67e-2d33-4f72-a4ea-d6601a4e5e57 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:36.516744 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ab24a67e-2d33-4f72-a4ea-d6601a4e5e57 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:36.517554 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ab24a67e-2d33-4f72-a4ea-d6601a4e5e57 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:36.518047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ab24a67e-2d33-4f72-a4ea-d6601a4e5e57 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:36.800949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=6 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:36.970197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5520ba0b-4bd5-4043-a0c5-999f48cc1dbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:36.979200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5520ba0b-4bd5-4043-a0c5-999f48cc1dbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.519602 Jun 10 13:05:36.992129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 37/67] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:36 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:36.998549 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ab24a67e-2d33-4f72-a4ea-d6601a4e5e57 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:37.006304 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ab24a67e-2d33-4f72-a4ea-d6601a4e5e57 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1501 microversion: 2.1 time: 0.505174 Jun 10 13:05:37.006879 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 31/68] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:36 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1501 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:37.454443 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-42597513-9570-4bce-a6fa-7af56b5293ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:37.455144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-42597513-9570-4bce-a6fa-7af56b5293ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:37.463131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-42597513-9570-4bce-a6fa-7af56b5293ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:37.463691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-42597513-9570-4bce-a6fa-7af56b5293ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:37.464569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-42597513-9570-4bce-a6fa-7af56b5293ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:37.465224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-42597513-9570-4bce-a6fa-7af56b5293ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:37.465795 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-0067ba47-b9fe-4b29-b29d-77531603ff7a req-dbedb8ec-54a9-4ceb-8d3e-a168ba42b204 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "db56898a-dff9-4cad-bf8d-f84b53247fc3", "tag": "31e1a9d0-3f1e-4221-a431-639f0e4a6ff4"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:05:37.474507 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-0067ba47-b9fe-4b29-b29d-77531603ff7a req-dbedb8ec-54a9-4ceb-8d3e-a168ba42b204 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:37.475581 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-0067ba47-b9fe-4b29-b29d-77531603ff7a req-dbedb8ec-54a9-4ceb-8d3e-a168ba42b204 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:37.494246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-0067ba47-b9fe-4b29-b29d-77531603ff7a req-dbedb8ec-54a9-4ceb-8d3e-a168ba42b204 service nova] Creating event network-changed:31e1a9d0-3f1e-4221-a431-639f0e4a6ff4 for instance db56898a-dff9-4cad-bf8d-f84b53247fc3 on hv-cinder-maste Jun 10 13:05:37.502133 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-0067ba47-b9fe-4b29-b29d-77531603ff7a req-dbedb8ec-54a9-4ceb-8d3e-a168ba42b204 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043610 Jun 10 13:05:37.503576 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 32/69] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:05:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:05:37.637014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-42597513-9570-4bce-a6fa-7af56b5293ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:37.647094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-42597513-9570-4bce-a6fa-7af56b5293ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:37.649007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-42597513-9570-4bce-a6fa-7af56b5293ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:37.650411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-42597513-9570-4bce-a6fa-7af56b5293ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.204354 Jun 10 13:05:37.651620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 38/70] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:37 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:37.994317 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=18 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:37.998543 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-397e8cfb-bd09-45d7-aeec-0bf914ba4525 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:37.999233 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-397e8cfb-bd09-45d7-aeec-0bf914ba4525 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:38.005576 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-397e8cfb-bd09-45d7-aeec-0bf914ba4525 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:38.006029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-397e8cfb-bd09-45d7-aeec-0bf914ba4525 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:38.006777 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-397e8cfb-bd09-45d7-aeec-0bf914ba4525 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:38.007205 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-397e8cfb-bd09-45d7-aeec-0bf914ba4525 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:38.177727 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-397e8cfb-bd09-45d7-aeec-0bf914ba4525 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:38.185147 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-397e8cfb-bd09-45d7-aeec-0bf914ba4525 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.191765 Jun 10 13:05:38.185717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 33/71] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:37 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:05:38.671077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3abe0e0b-575e-46b3-955c-8e73997be897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:38.672263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3abe0e0b-575e-46b3-955c-8e73997be897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:38.679734 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3abe0e0b-575e-46b3-955c-8e73997be897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:38.680182 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3abe0e0b-575e-46b3-955c-8e73997be897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:38.680983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3abe0e0b-575e-46b3-955c-8e73997be897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:38.681417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3abe0e0b-575e-46b3-955c-8e73997be897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:38.913034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3abe0e0b-575e-46b3-955c-8e73997be897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:38.931894 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cc5fccac-8e22-4706-a9e7-5ae3935f5984 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "983efab6-81dc-4561-9b40-c4431eefe0af"}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:05:38.935695 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cc5fccac-8e22-4706-a9e7-5ae3935f5984 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:38.941111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3abe0e0b-575e-46b3-955c-8e73997be897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:38.946580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3abe0e0b-575e-46b3-955c-8e73997be897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:38.949938 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3abe0e0b-575e-46b3-955c-8e73997be897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.285356 Jun 10 13:05:38.953534 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc5fccac-8e22-4706-a9e7-5ae3935f5984 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:38.954403 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc5fccac-8e22-4706-a9e7-5ae3935f5984 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:38.955181 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc5fccac-8e22-4706-a9e7-5ae3935f5984 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:38.955611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc5fccac-8e22-4706-a9e7-5ae3935f5984 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:38.964958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 39/72] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:38 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:39.205612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-90671246-69e7-4102-892e-6cd9f11a0f2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:39.206217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-90671246-69e7-4102-892e-6cd9f11a0f2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:39.212463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90671246-69e7-4102-892e-6cd9f11a0f2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:39.212942 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90671246-69e7-4102-892e-6cd9f11a0f2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:39.213735 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90671246-69e7-4102-892e-6cd9f11a0f2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:39.214862 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90671246-69e7-4102-892e-6cd9f11a0f2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:39.505352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-90671246-69e7-4102-892e-6cd9f11a0f2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:39.515099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-90671246-69e7-4102-892e-6cd9f11a0f2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.313202 Jun 10 13:05:39.526117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 40/73] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:39 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:05:39.975794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3e0276f0-3a3b-47bc-b191-2515362142b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:39.976431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3e0276f0-3a3b-47bc-b191-2515362142b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:39.982655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3e0276f0-3a3b-47bc-b191-2515362142b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:39.983097 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3e0276f0-3a3b-47bc-b191-2515362142b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:39.983859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3e0276f0-3a3b-47bc-b191-2515362142b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:39.984292 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3e0276f0-3a3b-47bc-b191-2515362142b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:40.177595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3e0276f0-3a3b-47bc-b191-2515362142b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:40.186281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3e0276f0-3a3b-47bc-b191-2515362142b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:40.187884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3e0276f0-3a3b-47bc-b191-2515362142b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:40.189226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3e0276f0-3a3b-47bc-b191-2515362142b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.217612 Jun 10 13:05:40.189789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 41/74] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:39 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:40.453844 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cc5fccac-8e22-4706-a9e7-5ae3935f5984 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "POST /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.528691 Jun 10 13:05:40.455199 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 34/75] 10.222.0.25 () {60 vars in 1322 bytes} [Fri Jun 10 13:05:38 2022] POST /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1/os-volume_attachments => generated 194 bytes in 1531 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:05:40.533006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d13ae2e5-9a61-43f4-bf2f-3b6a9df8c42b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:40.533701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d13ae2e5-9a61-43f4-bf2f-3b6a9df8c42b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:40.540958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d13ae2e5-9a61-43f4-bf2f-3b6a9df8c42b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:40.541510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d13ae2e5-9a61-43f4-bf2f-3b6a9df8c42b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:40.542568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d13ae2e5-9a61-43f4-bf2f-3b6a9df8c42b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:40.543159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d13ae2e5-9a61-43f4-bf2f-3b6a9df8c42b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:40.694694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d13ae2e5-9a61-43f4-bf2f-3b6a9df8c42b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:40.706268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d13ae2e5-9a61-43f4-bf2f-3b6a9df8c42b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.177670 Jun 10 13:05:40.712084 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 42/76] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:40 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:41.207886 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-04c70fc3-4782-4da1-9fe6-6c003791b759 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:41.208585 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-04c70fc3-4782-4da1-9fe6-6c003791b759 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:41.216161 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04c70fc3-4782-4da1-9fe6-6c003791b759 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:41.216612 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04c70fc3-4782-4da1-9fe6-6c003791b759 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:41.217429 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04c70fc3-4782-4da1-9fe6-6c003791b759 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:41.217865 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04c70fc3-4782-4da1-9fe6-6c003791b759 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:41.355959 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-04c70fc3-4782-4da1-9fe6-6c003791b759 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:41.364112 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-04c70fc3-4782-4da1-9fe6-6c003791b759 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:41.365652 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-04c70fc3-4782-4da1-9fe6-6c003791b759 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:41.366753 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-04c70fc3-4782-4da1-9fe6-6c003791b759 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.163114 Jun 10 13:05:41.367311 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 35/77] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:41 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:41.723336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fa31ee5f-0a85-459c-a056-9a64a59cec19 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:41.723939 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fa31ee5f-0a85-459c-a056-9a64a59cec19 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:41.734467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fa31ee5f-0a85-459c-a056-9a64a59cec19 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:41.734920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fa31ee5f-0a85-459c-a056-9a64a59cec19 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:41.735697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fa31ee5f-0a85-459c-a056-9a64a59cec19 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:41.736142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fa31ee5f-0a85-459c-a056-9a64a59cec19 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:41.897840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fa31ee5f-0a85-459c-a056-9a64a59cec19 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:41.898924 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fa31ee5f-0a85-459c-a056-9a64a59cec19 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.179405 Jun 10 13:05:41.900794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 43/78] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:41 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:42.386415 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7cc47ef2-f05d-4365-834a-9c163e486d0c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:42.387328 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7cc47ef2-f05d-4365-834a-9c163e486d0c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:42.394419 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7cc47ef2-f05d-4365-834a-9c163e486d0c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:42.394873 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7cc47ef2-f05d-4365-834a-9c163e486d0c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:42.395664 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7cc47ef2-f05d-4365-834a-9c163e486d0c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:42.396104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7cc47ef2-f05d-4365-834a-9c163e486d0c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:42.537736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7cc47ef2-f05d-4365-834a-9c163e486d0c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:42.547106 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7cc47ef2-f05d-4365-834a-9c163e486d0c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:42.548786 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7cc47ef2-f05d-4365-834a-9c163e486d0c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:42.550019 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7cc47ef2-f05d-4365-834a-9c163e486d0c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.168245 Jun 10 13:05:42.550685 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 36/79] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:42 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:42.921295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dfd5d479-52b0-4a60-a8bc-e08e88be1b28 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:42.921943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dfd5d479-52b0-4a60-a8bc-e08e88be1b28 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:42.928215 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dfd5d479-52b0-4a60-a8bc-e08e88be1b28 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:42.928672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dfd5d479-52b0-4a60-a8bc-e08e88be1b28 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:42.929487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dfd5d479-52b0-4a60-a8bc-e08e88be1b28 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:42.929925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dfd5d479-52b0-4a60-a8bc-e08e88be1b28 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:43.101329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dfd5d479-52b0-4a60-a8bc-e08e88be1b28 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:43.110693 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dfd5d479-52b0-4a60-a8bc-e08e88be1b28 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.197182 Jun 10 13:05:43.111279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 44/80] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:42 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:43.570410 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-20f09b98-903c-47b5-b669-33e2162116ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:43.571146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-20f09b98-903c-47b5-b669-33e2162116ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:43.581749 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-20f09b98-903c-47b5-b669-33e2162116ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:43.582468 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-20f09b98-903c-47b5-b669-33e2162116ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:43.584064 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-20f09b98-903c-47b5-b669-33e2162116ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:43.585194 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-20f09b98-903c-47b5-b669-33e2162116ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:43.754198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-20f09b98-903c-47b5-b669-33e2162116ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:43.767068 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-20f09b98-903c-47b5-b669-33e2162116ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:43.768585 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-20f09b98-903c-47b5-b669-33e2162116ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:43.769754 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-20f09b98-903c-47b5-b669-33e2162116ce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.204702 Jun 10 13:05:43.770472 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 37/81] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:43 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:44.128353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-453bf0ab-7cc0-45ef-8d2f-4bd0592d9be8 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:44.129565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-453bf0ab-7cc0-45ef-8d2f-4bd0592d9be8 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:44.140413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-453bf0ab-7cc0-45ef-8d2f-4bd0592d9be8 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:44.141268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-453bf0ab-7cc0-45ef-8d2f-4bd0592d9be8 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:44.142049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-453bf0ab-7cc0-45ef-8d2f-4bd0592d9be8 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:44.142631 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-453bf0ab-7cc0-45ef-8d2f-4bd0592d9be8 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:44.146983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:44.273921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-453bf0ab-7cc0-45ef-8d2f-4bd0592d9be8 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:44.281860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-453bf0ab-7cc0-45ef-8d2f-4bd0592d9be8 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.157387 Jun 10 13:05:44.282519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 45/82] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:44 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:44.793628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ad9abb90-68d2-4c7c-bd18-3485b8dca7ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:44.794385 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ad9abb90-68d2-4c7c-bd18-3485b8dca7ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:44.801505 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ad9abb90-68d2-4c7c-bd18-3485b8dca7ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:44.802212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ad9abb90-68d2-4c7c-bd18-3485b8dca7ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:44.803119 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ad9abb90-68d2-4c7c-bd18-3485b8dca7ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:44.803700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ad9abb90-68d2-4c7c-bd18-3485b8dca7ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:44.891462 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-b4f73d36-bd0c-47bb-8707-ca3c6cc42290 req-70e33d1a-4719-43c3-91c1-5da1c3ead340 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9e403cf8-5aee-49f3-bbbc-600263bd148a", "name": "network-vif-plugged", "status": "completed", "tag": "9615dc48-0838-44bb-9298-f3df821b26c7"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:05:44.906477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-b4f73d36-bd0c-47bb-8707-ca3c6cc42290 req-70e33d1a-4719-43c3-91c1-5da1c3ead340 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:44.908181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-b4f73d36-bd0c-47bb-8707-ca3c6cc42290 req-70e33d1a-4719-43c3-91c1-5da1c3ead340 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:44.931037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-b4f73d36-bd0c-47bb-8707-ca3c6cc42290 req-70e33d1a-4719-43c3-91c1-5da1c3ead340 service nova] Creating event network-vif-plugged:9615dc48-0838-44bb-9298-f3df821b26c7 for instance 9e403cf8-5aee-49f3-bbbc-600263bd148a on hv-cinder-maste Jun 10 13:05:44.936983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-b4f73d36-bd0c-47bb-8707-ca3c6cc42290 req-70e33d1a-4719-43c3-91c1-5da1c3ead340 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050122 Jun 10 13:05:44.937935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 46/83] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:05:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:05:44.962619 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ad9abb90-68d2-4c7c-bd18-3485b8dca7ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:44.967509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ad9abb90-68d2-4c7c-bd18-3485b8dca7ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:44.969033 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ad9abb90-68d2-4c7c-bd18-3485b8dca7ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:44.970134 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ad9abb90-68d2-4c7c-bd18-3485b8dca7ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.181104 Jun 10 13:05:44.970682 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 38/84] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:44 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:45.296645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:45.301864 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b3f2a61a-e2f5-4ce0-ac11-9aaa26d1764c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:45.302632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b3f2a61a-e2f5-4ce0-ac11-9aaa26d1764c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:45.331309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3f2a61a-e2f5-4ce0-ac11-9aaa26d1764c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:45.331309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3f2a61a-e2f5-4ce0-ac11-9aaa26d1764c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:45.331309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3f2a61a-e2f5-4ce0-ac11-9aaa26d1764c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:45.331309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3f2a61a-e2f5-4ce0-ac11-9aaa26d1764c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:45.492540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b3f2a61a-e2f5-4ce0-ac11-9aaa26d1764c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:45.501722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b3f2a61a-e2f5-4ce0-ac11-9aaa26d1764c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.205975 Jun 10 13:05:45.502740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 47/85] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:45 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:45.989580 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3abc4b35-9015-45f7-8486-3fd7b9951974 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:45.993112 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3abc4b35-9015-45f7-8486-3fd7b9951974 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:45.997717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3abc4b35-9015-45f7-8486-3fd7b9951974 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:45.998163 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3abc4b35-9015-45f7-8486-3fd7b9951974 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:45.998924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3abc4b35-9015-45f7-8486-3fd7b9951974 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:45.999514 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3abc4b35-9015-45f7-8486-3fd7b9951974 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:46.180893 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3abc4b35-9015-45f7-8486-3fd7b9951974 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:46.188752 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3abc4b35-9015-45f7-8486-3fd7b9951974 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:46.190292 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3abc4b35-9015-45f7-8486-3fd7b9951974 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:46.191434 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3abc4b35-9015-45f7-8486-3fd7b9951974 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.205928 Jun 10 13:05:46.191995 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 39/86] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:45 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:46.519801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4d826b82-f4f2-4c41-9354-d8e448925fbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:46.520409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4d826b82-f4f2-4c41-9354-d8e448925fbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:46.526657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4d826b82-f4f2-4c41-9354-d8e448925fbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:46.527101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4d826b82-f4f2-4c41-9354-d8e448925fbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:46.527858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4d826b82-f4f2-4c41-9354-d8e448925fbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:46.528291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4d826b82-f4f2-4c41-9354-d8e448925fbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:46.664621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4d826b82-f4f2-4c41-9354-d8e448925fbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:46.673205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4d826b82-f4f2-4c41-9354-d8e448925fbd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.157479 Jun 10 13:05:46.674018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 48/87] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:46 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:46.951611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-36d38208-cc4a-4895-8105-3ee85d40a620 req-35b56601-8125-447f-b98d-12f284af2a68 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "db56898a-dff9-4cad-bf8d-f84b53247fc3", "name": "network-vif-plugged", "status": "completed", "tag": "31e1a9d0-3f1e-4221-a431-639f0e4a6ff4"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:05:46.959693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-36d38208-cc4a-4895-8105-3ee85d40a620 req-35b56601-8125-447f-b98d-12f284af2a68 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:46.960129 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-36d38208-cc4a-4895-8105-3ee85d40a620 req-35b56601-8125-447f-b98d-12f284af2a68 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:46.971038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-36d38208-cc4a-4895-8105-3ee85d40a620 req-35b56601-8125-447f-b98d-12f284af2a68 service nova] Creating event network-vif-plugged:31e1a9d0-3f1e-4221-a431-639f0e4a6ff4 for instance db56898a-dff9-4cad-bf8d-f84b53247fc3 on hv-cinder-maste Jun 10 13:05:46.976499 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-36d38208-cc4a-4895-8105-3ee85d40a620 req-35b56601-8125-447f-b98d-12f284af2a68 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028381 Jun 10 13:05:46.977130 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 40/88] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:05:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:05:47.209330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d781b532-141f-49db-8c7a-76a4365ced66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:47.209935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d781b532-141f-49db-8c7a-76a4365ced66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:47.216585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d781b532-141f-49db-8c7a-76a4365ced66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:47.217073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d781b532-141f-49db-8c7a-76a4365ced66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:47.217838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d781b532-141f-49db-8c7a-76a4365ced66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:47.218269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d781b532-141f-49db-8c7a-76a4365ced66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:47.394110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d781b532-141f-49db-8c7a-76a4365ced66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:47.402035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d781b532-141f-49db-8c7a-76a4365ced66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:47.404103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d781b532-141f-49db-8c7a-76a4365ced66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:47.405286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d781b532-141f-49db-8c7a-76a4365ced66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.199973 Jun 10 13:05:47.405836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 49/89] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:47 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:47.695795 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-48e83b8b-a664-4496-a891-84c38edd8be2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:47.695795 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-48e83b8b-a664-4496-a891-84c38edd8be2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:47.699804 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-48e83b8b-a664-4496-a891-84c38edd8be2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:47.700262 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-48e83b8b-a664-4496-a891-84c38edd8be2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:47.701057 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-48e83b8b-a664-4496-a891-84c38edd8be2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:47.701489 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-48e83b8b-a664-4496-a891-84c38edd8be2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:47.843223 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-48e83b8b-a664-4496-a891-84c38edd8be2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:47.850659 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-48e83b8b-a664-4496-a891-84c38edd8be2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.163624 Jun 10 13:05:47.853713 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 41/90] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:47 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:48.423178 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b11052b2-309d-473e-a245-d2eafa1116c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:48.423773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b11052b2-309d-473e-a245-d2eafa1116c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:48.430014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b11052b2-309d-473e-a245-d2eafa1116c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:48.430454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b11052b2-309d-473e-a245-d2eafa1116c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:48.431210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b11052b2-309d-473e-a245-d2eafa1116c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:48.431635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b11052b2-309d-473e-a245-d2eafa1116c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:48.580695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b11052b2-309d-473e-a245-d2eafa1116c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:48.589049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b11052b2-309d-473e-a245-d2eafa1116c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:48.590558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b11052b2-309d-473e-a245-d2eafa1116c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:48.591640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b11052b2-309d-473e-a245-d2eafa1116c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.172318 Jun 10 13:05:48.592188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 50/91] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:48 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:48.868237 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e7638725-7160-4bfd-ad66-6bfac3ece446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:48.868963 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e7638725-7160-4bfd-ad66-6bfac3ece446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:48.876124 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e7638725-7160-4bfd-ad66-6bfac3ece446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:48.876671 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e7638725-7160-4bfd-ad66-6bfac3ece446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:48.877494 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e7638725-7160-4bfd-ad66-6bfac3ece446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:48.877929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e7638725-7160-4bfd-ad66-6bfac3ece446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:49.013274 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e7638725-7160-4bfd-ad66-6bfac3ece446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:49.020836 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e7638725-7160-4bfd-ad66-6bfac3ece446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.156301 Jun 10 13:05:49.034512 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 42/92] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:48 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:49.609486 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2d960fa6-600f-446a-88f5-f4cdab442874 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:49.610059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2d960fa6-600f-446a-88f5-f4cdab442874 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:49.616542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d960fa6-600f-446a-88f5-f4cdab442874 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:49.617024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d960fa6-600f-446a-88f5-f4cdab442874 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:49.617778 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d960fa6-600f-446a-88f5-f4cdab442874 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:49.618207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d960fa6-600f-446a-88f5-f4cdab442874 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:49.820473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2d960fa6-600f-446a-88f5-f4cdab442874 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:49.826869 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2d960fa6-600f-446a-88f5-f4cdab442874 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:49.828585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2d960fa6-600f-446a-88f5-f4cdab442874 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:49.829814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2d960fa6-600f-446a-88f5-f4cdab442874 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.224200 Jun 10 13:05:49.841102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 51/93] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:49 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:50.037400 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-66923574-dc33-4aa9-a584-b752ee907950 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:50.038013 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-66923574-dc33-4aa9-a584-b752ee907950 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:50.053877 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66923574-dc33-4aa9-a584-b752ee907950 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:50.054586 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66923574-dc33-4aa9-a584-b752ee907950 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:50.055401 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66923574-dc33-4aa9-a584-b752ee907950 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:50.055868 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66923574-dc33-4aa9-a584-b752ee907950 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:50.216179 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-66923574-dc33-4aa9-a584-b752ee907950 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:50.224946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-66923574-dc33-4aa9-a584-b752ee907950 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.191243 Jun 10 13:05:50.226080 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 43/94] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:50 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:50.847433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7a2c33e1-756e-45a5-807c-4e0aa2b08317 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:50.848176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7a2c33e1-756e-45a5-807c-4e0aa2b08317 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:50.854535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a2c33e1-756e-45a5-807c-4e0aa2b08317 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:50.855034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a2c33e1-756e-45a5-807c-4e0aa2b08317 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:50.856389 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a2c33e1-756e-45a5-807c-4e0aa2b08317 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:50.856852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a2c33e1-756e-45a5-807c-4e0aa2b08317 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:50.992630 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7a2c33e1-756e-45a5-807c-4e0aa2b08317 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:51.000829 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7a2c33e1-756e-45a5-807c-4e0aa2b08317 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:51.002532 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7a2c33e1-756e-45a5-807c-4e0aa2b08317 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:51.004211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7a2c33e1-756e-45a5-807c-4e0aa2b08317 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.160728 Jun 10 13:05:51.004800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 52/95] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:50 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:51.242581 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d92f17f9-12d5-4cf1-8afd-6aa0e81d0bf2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:51.243189 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d92f17f9-12d5-4cf1-8afd-6aa0e81d0bf2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:51.250321 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d92f17f9-12d5-4cf1-8afd-6aa0e81d0bf2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:51.250779 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d92f17f9-12d5-4cf1-8afd-6aa0e81d0bf2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:51.251541 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d92f17f9-12d5-4cf1-8afd-6aa0e81d0bf2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:51.251973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d92f17f9-12d5-4cf1-8afd-6aa0e81d0bf2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:51.398167 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d92f17f9-12d5-4cf1-8afd-6aa0e81d0bf2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:51.400477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:51.406989 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d92f17f9-12d5-4cf1-8afd-6aa0e81d0bf2 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.168316 Jun 10 13:05:51.407892 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 44/96] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:51 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:52.022512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5c0d4382-e948-4089-aee7-5992bb315dc4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:52.023121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5c0d4382-e948-4089-aee7-5992bb315dc4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:52.029683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c0d4382-e948-4089-aee7-5992bb315dc4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:52.030134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c0d4382-e948-4089-aee7-5992bb315dc4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:52.030898 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c0d4382-e948-4089-aee7-5992bb315dc4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:52.031330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c0d4382-e948-4089-aee7-5992bb315dc4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:52.165408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5c0d4382-e948-4089-aee7-5992bb315dc4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:52.173310 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5c0d4382-e948-4089-aee7-5992bb315dc4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:52.174806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5c0d4382-e948-4089-aee7-5992bb315dc4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:52.175905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5c0d4382-e948-4089-aee7-5992bb315dc4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.157298 Jun 10 13:05:52.176458 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 53/97] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:52 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:52.423859 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-08aae780-dc0f-4e4d-b811-5ac42ed149fe tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:52.424470 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-08aae780-dc0f-4e4d-b811-5ac42ed149fe tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:52.431434 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-08aae780-dc0f-4e4d-b811-5ac42ed149fe tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:52.431878 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-08aae780-dc0f-4e4d-b811-5ac42ed149fe tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:52.432635 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-08aae780-dc0f-4e4d-b811-5ac42ed149fe tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:52.433797 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-08aae780-dc0f-4e4d-b811-5ac42ed149fe tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:52.575358 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-08aae780-dc0f-4e4d-b811-5ac42ed149fe tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:52.583369 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-08aae780-dc0f-4e4d-b811-5ac42ed149fe tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1363 microversion: 2.1 time: 0.163738 Jun 10 13:05:52.584477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 45/98] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:52 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1363 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:53.194673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9da7d0aa-a50f-46eb-8e5a-fced88e0a6df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:53.196318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9da7d0aa-a50f-46eb-8e5a-fced88e0a6df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:53.203142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9da7d0aa-a50f-46eb-8e5a-fced88e0a6df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:53.203585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9da7d0aa-a50f-46eb-8e5a-fced88e0a6df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:53.204347 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9da7d0aa-a50f-46eb-8e5a-fced88e0a6df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:53.204844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9da7d0aa-a50f-46eb-8e5a-fced88e0a6df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:53.369668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9da7d0aa-a50f-46eb-8e5a-fced88e0a6df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:53.389976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9da7d0aa-a50f-46eb-8e5a-fced88e0a6df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:53.391580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9da7d0aa-a50f-46eb-8e5a-fced88e0a6df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:53.393058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9da7d0aa-a50f-46eb-8e5a-fced88e0a6df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.202428 Jun 10 13:05:53.393619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 54/99] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:53 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:53.601000 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d9b3e4fc-0a40-426c-a87f-7123010640b3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:53.602264 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d9b3e4fc-0a40-426c-a87f-7123010640b3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:53.610892 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d9b3e4fc-0a40-426c-a87f-7123010640b3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:53.611587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d9b3e4fc-0a40-426c-a87f-7123010640b3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:53.612519 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d9b3e4fc-0a40-426c-a87f-7123010640b3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:53.613668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d9b3e4fc-0a40-426c-a87f-7123010640b3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:53.752871 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d9b3e4fc-0a40-426c-a87f-7123010640b3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:53.761261 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d9b3e4fc-0a40-426c-a87f-7123010640b3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1549 microversion: 2.1 time: 0.164145 Jun 10 13:05:53.762116 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 46/100] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:53 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1549 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:53.778804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2b35b3c2-2aaa-4652-9218-97d3c820d4fc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:53.779290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2b35b3c2-2aaa-4652-9218-97d3c820d4fc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:53.785349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2b35b3c2-2aaa-4652-9218-97d3c820d4fc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:53.785799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2b35b3c2-2aaa-4652-9218-97d3c820d4fc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:53.786554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2b35b3c2-2aaa-4652-9218-97d3c820d4fc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:53.787004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2b35b3c2-2aaa-4652-9218-97d3c820d4fc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:53.908149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2b35b3c2-2aaa-4652-9218-97d3c820d4fc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.133756 Jun 10 13:05:53.909177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 55/101] 10.222.0.25 () {58 vars in 1274 bytes} [Fri Jun 10 13:05:53 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/os-interface => generated 284 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:05:54.411429 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-29442035-6a23-48b1-aa2b-f8826fbbb2c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:54.412044 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-29442035-6a23-48b1-aa2b-f8826fbbb2c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:54.434510 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29442035-6a23-48b1-aa2b-f8826fbbb2c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:54.435221 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29442035-6a23-48b1-aa2b-f8826fbbb2c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:54.437489 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29442035-6a23-48b1-aa2b-f8826fbbb2c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:54.444017 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29442035-6a23-48b1-aa2b-f8826fbbb2c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:54.748466 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-29442035-6a23-48b1-aa2b-f8826fbbb2c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:54.756421 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-29442035-6a23-48b1-aa2b-f8826fbbb2c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:54.757966 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-29442035-6a23-48b1-aa2b-f8826fbbb2c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:54.759056 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-29442035-6a23-48b1-aa2b-f8826fbbb2c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.351510 Jun 10 13:05:54.759920 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 47/102] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:54 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:55.547855 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-060bf609-b58b-4080-812e-19e7d741029a req-ba67e5a6-9312-482f-b48a-80860ace11d8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9e403cf8-5aee-49f3-bbbc-600263bd148a", "tag": "9615dc48-0838-44bb-9298-f3df821b26c7"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:05:55.558902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-060bf609-b58b-4080-812e-19e7d741029a req-ba67e5a6-9312-482f-b48a-80860ace11d8 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:55.560654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-060bf609-b58b-4080-812e-19e7d741029a req-ba67e5a6-9312-482f-b48a-80860ace11d8 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:55.580909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-060bf609-b58b-4080-812e-19e7d741029a req-ba67e5a6-9312-482f-b48a-80860ace11d8 service nova] Creating event network-changed:9615dc48-0838-44bb-9298-f3df821b26c7 for instance 9e403cf8-5aee-49f3-bbbc-600263bd148a on hv-cinder-maste Jun 10 13:05:55.580909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-060bf609-b58b-4080-812e-19e7d741029a req-ba67e5a6-9312-482f-b48a-80860ace11d8 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037163 Jun 10 13:05:55.585326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 56/103] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:05:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:05:55.779986 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e3db36dc-d8d5-4da6-b1e3-73446e186549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:55.781653 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e3db36dc-d8d5-4da6-b1e3-73446e186549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:55.788880 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e3db36dc-d8d5-4da6-b1e3-73446e186549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:55.789717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e3db36dc-d8d5-4da6-b1e3-73446e186549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:55.790139 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e3db36dc-d8d5-4da6-b1e3-73446e186549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:55.790582 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e3db36dc-d8d5-4da6-b1e3-73446e186549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:55.985176 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e3db36dc-d8d5-4da6-b1e3-73446e186549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:55.993860 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e3db36dc-d8d5-4da6-b1e3-73446e186549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:55.995606 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e3db36dc-d8d5-4da6-b1e3-73446e186549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:55.996721 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e3db36dc-d8d5-4da6-b1e3-73446e186549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.221627 Jun 10 13:05:55.997293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 48/104] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:55 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:05:57.014923 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0919e0b6-aed6-401c-b1b1-27539ae6417d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:57.016730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0919e0b6-aed6-401c-b1b1-27539ae6417d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:57.021828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0919e0b6-aed6-401c-b1b1-27539ae6417d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:57.022269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0919e0b6-aed6-401c-b1b1-27539ae6417d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:57.023014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0919e0b6-aed6-401c-b1b1-27539ae6417d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:57.023439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0919e0b6-aed6-401c-b1b1-27539ae6417d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:57.162139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0919e0b6-aed6-401c-b1b1-27539ae6417d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:57.170565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0919e0b6-aed6-401c-b1b1-27539ae6417d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:57.172052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0919e0b6-aed6-401c-b1b1-27539ae6417d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:57.173183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0919e0b6-aed6-401c-b1b1-27539ae6417d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.162426 Jun 10 13:05:57.173751 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 57/105] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:57 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:57.820637 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7f076d99-4dd1-42d3-a504-f2d2a69c0f09 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:57.821224 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7f076d99-4dd1-42d3-a504-f2d2a69c0f09 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:57.828108 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7f076d99-4dd1-42d3-a504-f2d2a69c0f09 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:57.828550 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7f076d99-4dd1-42d3-a504-f2d2a69c0f09 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:57.829344 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7f076d99-4dd1-42d3-a504-f2d2a69c0f09 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:57.829783 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7f076d99-4dd1-42d3-a504-f2d2a69c0f09 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:57.855707 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7f076d99-4dd1-42d3-a504-f2d2a69c0f09 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Going to try to terminate instance {{(pid=118093) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:05:58.149904 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7f076d99-4dd1-42d3-a504-f2d2a69c0f09 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "DELETE /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 204 len: 0 microversion: 2.1 time: 0.332502 Jun 10 13:05:58.150632 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 49/106] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:05:57 2022] DELETE /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jun 10 13:05:58.159504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b5ac1351-9230-48ab-a696-3b86f01ff09b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:58.160106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b5ac1351-9230-48ab-a696-3b86f01ff09b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:58.166172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5ac1351-9230-48ab-a696-3b86f01ff09b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:58.166615 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5ac1351-9230-48ab-a696-3b86f01ff09b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:58.168084 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5ac1351-9230-48ab-a696-3b86f01ff09b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:58.168515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5ac1351-9230-48ab-a696-3b86f01ff09b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:58.193376 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6f237a2a-35f9-4a19-9161-4389eceac2fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:58.194293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6f237a2a-35f9-4a19-9161-4389eceac2fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:58.200926 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6f237a2a-35f9-4a19-9161-4389eceac2fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:58.201477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6f237a2a-35f9-4a19-9161-4389eceac2fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:58.202502 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6f237a2a-35f9-4a19-9161-4389eceac2fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:58.203018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6f237a2a-35f9-4a19-9161-4389eceac2fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:58.317617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b5ac1351-9230-48ab-a696-3b86f01ff09b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:58.340581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b5ac1351-9230-48ab-a696-3b86f01ff09b tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1553 microversion: 2.1 time: 0.185090 Jun 10 13:05:58.341409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 58/107] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:58 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1553 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:58.349372 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6f237a2a-35f9-4a19-9161-4389eceac2fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:58.357943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6f237a2a-35f9-4a19-9161-4389eceac2fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:58.359413 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6f237a2a-35f9-4a19-9161-4389eceac2fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:58.360543 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6f237a2a-35f9-4a19-9161-4389eceac2fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.172870 Jun 10 13:05:58.361127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 50/108] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:58 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:05:59.361871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e0894f4d-74f8-44bf-8a6a-177d021debeb tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:59.362750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e0894f4d-74f8-44bf-8a6a-177d021debeb tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:59.369586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0894f4d-74f8-44bf-8a6a-177d021debeb tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:59.370041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0894f4d-74f8-44bf-8a6a-177d021debeb tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:59.370810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0894f4d-74f8-44bf-8a6a-177d021debeb tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:59.371239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0894f4d-74f8-44bf-8a6a-177d021debeb tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:59.383680 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f404a49a-c26d-461f-9322-7304711e5f66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:05:59.384818 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f404a49a-c26d-461f-9322-7304711e5f66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:05:59.393356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f404a49a-c26d-461f-9322-7304711e5f66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:59.397082 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f404a49a-c26d-461f-9322-7304711e5f66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:59.397082 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f404a49a-c26d-461f-9322-7304711e5f66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:05:59.397082 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f404a49a-c26d-461f-9322-7304711e5f66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:05:59.504159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e0894f4d-74f8-44bf-8a6a-177d021debeb tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:59.516379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e0894f4d-74f8-44bf-8a6a-177d021debeb tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1553 microversion: 2.1 time: 0.157871 Jun 10 13:05:59.517599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 59/109] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:05:59 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1553 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:05:59.545626 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f404a49a-c26d-461f-9322-7304711e5f66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:59.553728 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f404a49a-c26d-461f-9322-7304711e5f66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:59.555213 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f404a49a-c26d-461f-9322-7304711e5f66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:05:59.556305 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f404a49a-c26d-461f-9322-7304711e5f66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.179882 Jun 10 13:05:59.556872 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 51/110] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:05:59 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:06:00.533627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9719a6b6-8770-4825-a4d9-4f5f4959e048 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:00.534235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9719a6b6-8770-4825-a4d9-4f5f4959e048 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:00.540222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9719a6b6-8770-4825-a4d9-4f5f4959e048 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:00.540668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9719a6b6-8770-4825-a4d9-4f5f4959e048 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:00.541516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9719a6b6-8770-4825-a4d9-4f5f4959e048 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:00.545781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9719a6b6-8770-4825-a4d9-4f5f4959e048 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:00.574597 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-968c7500-b745-4395-9177-006249c426d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:00.575196 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-968c7500-b745-4395-9177-006249c426d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:00.581889 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-968c7500-b745-4395-9177-006249c426d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:00.582325 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-968c7500-b745-4395-9177-006249c426d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:00.583086 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-968c7500-b745-4395-9177-006249c426d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:00.583512 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-968c7500-b745-4395-9177-006249c426d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:00.673316 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9719a6b6-8770-4825-a4d9-4f5f4959e048 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:00.681623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9719a6b6-8770-4825-a4d9-4f5f4959e048 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1553 microversion: 2.1 time: 0.151743 Jun 10 13:06:00.682210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 60/111] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:00 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1553 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:00.717872 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-968c7500-b745-4395-9177-006249c426d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:00.725789 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-968c7500-b745-4395-9177-006249c426d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:00.727312 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-968c7500-b745-4395-9177-006249c426d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:00.728445 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-968c7500-b745-4395-9177-006249c426d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.158318 Jun 10 13:06:00.729036 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 52/112] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:06:00 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:06:01.699597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d0331653-d910-4b11-8a5b-a81423e33ba7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:01.700200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d0331653-d910-4b11-8a5b-a81423e33ba7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:01.706346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d0331653-d910-4b11-8a5b-a81423e33ba7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:01.706845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d0331653-d910-4b11-8a5b-a81423e33ba7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:01.708153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d0331653-d910-4b11-8a5b-a81423e33ba7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:01.709587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d0331653-d910-4b11-8a5b-a81423e33ba7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:01.746787 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-37669e42-ed27-4188-8eda-93022d3961be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:01.747424 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-37669e42-ed27-4188-8eda-93022d3961be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:01.754639 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37669e42-ed27-4188-8eda-93022d3961be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:01.754639 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37669e42-ed27-4188-8eda-93022d3961be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:01.755456 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37669e42-ed27-4188-8eda-93022d3961be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:01.755891 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37669e42-ed27-4188-8eda-93022d3961be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:01.857336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d0331653-d910-4b11-8a5b-a81423e33ba7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:01.865576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d0331653-d910-4b11-8a5b-a81423e33ba7 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1553 microversion: 2.1 time: 0.170384 Jun 10 13:06:01.866140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 61/113] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:01 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1553 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:01.892387 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-37669e42-ed27-4188-8eda-93022d3961be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:01.900260 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-37669e42-ed27-4188-8eda-93022d3961be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:01.901834 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-37669e42-ed27-4188-8eda-93022d3961be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:01.902926 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-37669e42-ed27-4188-8eda-93022d3961be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.160095 Jun 10 13:06:01.915649 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 53/114] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:06:01 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:06:02.883654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0f4b86d8-1ff3-480a-947f-9ce9c8a1af6d tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:02.884255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0f4b86d8-1ff3-480a-947f-9ce9c8a1af6d tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:02.886286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=70 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:02.891424 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f4b86d8-1ff3-480a-947f-9ce9c8a1af6d tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:02.891874 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f4b86d8-1ff3-480a-947f-9ce9c8a1af6d tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:02.892638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f4b86d8-1ff3-480a-947f-9ce9c8a1af6d tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:02.893135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f4b86d8-1ff3-480a-947f-9ce9c8a1af6d tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:02.921212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ef765351-381c-430b-9bb6-070ef54e13ea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:02.921821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ef765351-381c-430b-9bb6-070ef54e13ea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:02.928256 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef765351-381c-430b-9bb6-070ef54e13ea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:02.928702 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef765351-381c-430b-9bb6-070ef54e13ea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:02.929495 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef765351-381c-430b-9bb6-070ef54e13ea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:02.929922 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef765351-381c-430b-9bb6-070ef54e13ea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:03.024015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0f4b86d8-1ff3-480a-947f-9ce9c8a1af6d tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:03.032660 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0f4b86d8-1ff3-480a-947f-9ce9c8a1af6d tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1553 microversion: 2.1 time: 0.152796 Jun 10 13:06:03.033595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 62/115] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:02 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1553 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:03.076449 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ef765351-381c-430b-9bb6-070ef54e13ea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:03.084827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ef765351-381c-430b-9bb6-070ef54e13ea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:03.086643 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ef765351-381c-430b-9bb6-070ef54e13ea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:03.088008 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ef765351-381c-430b-9bb6-070ef54e13ea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.171091 Jun 10 13:06:03.088635 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 54/116] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:06:02 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:06:04.049863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-acecf01a-5e51-4fa1-99f5-3242db605901 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:04.050473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-acecf01a-5e51-4fa1-99f5-3242db605901 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:04.056604 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acecf01a-5e51-4fa1-99f5-3242db605901 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:04.060381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acecf01a-5e51-4fa1-99f5-3242db605901 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:04.060381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acecf01a-5e51-4fa1-99f5-3242db605901 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:04.060381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acecf01a-5e51-4fa1-99f5-3242db605901 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:04.073777 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:04.107019 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a3d98973-e9c1-4e96-8107-cc406a94ee4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:04.108009 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a3d98973-e9c1-4e96-8107-cc406a94ee4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:04.115330 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3d98973-e9c1-4e96-8107-cc406a94ee4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:04.116009 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3d98973-e9c1-4e96-8107-cc406a94ee4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:04.117158 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3d98973-e9c1-4e96-8107-cc406a94ee4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:04.117794 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3d98973-e9c1-4e96-8107-cc406a94ee4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:04.198537 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-acecf01a-5e51-4fa1-99f5-3242db605901 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:04.205151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-acecf01a-5e51-4fa1-99f5-3242db605901 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1553 microversion: 2.1 time: 0.159124 Jun 10 13:06:04.205726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 63/117] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:04 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1553 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:04.259026 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a3d98973-e9c1-4e96-8107-cc406a94ee4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:04.266895 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a3d98973-e9c1-4e96-8107-cc406a94ee4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:04.268377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a3d98973-e9c1-4e96-8107-cc406a94ee4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:04.269499 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a3d98973-e9c1-4e96-8107-cc406a94ee4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.167160 Jun 10 13:06:04.270056 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 55/118] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:06:04 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:06:05.222595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-55bec205-5113-4bf4-b538-0b70377ab8a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:05.223199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-55bec205-5113-4bf4-b538-0b70377ab8a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:05.229066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55bec205-5113-4bf4-b538-0b70377ab8a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:05.230017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55bec205-5113-4bf4-b538-0b70377ab8a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:05.230767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55bec205-5113-4bf4-b538-0b70377ab8a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:05.231221 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55bec205-5113-4bf4-b538-0b70377ab8a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:05.287292 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4227a0df-53bd-464e-ae8a-5ba55ed4b917 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:05.287888 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4227a0df-53bd-464e-ae8a-5ba55ed4b917 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:05.294355 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4227a0df-53bd-464e-ae8a-5ba55ed4b917 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:05.294803 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4227a0df-53bd-464e-ae8a-5ba55ed4b917 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:05.295562 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4227a0df-53bd-464e-ae8a-5ba55ed4b917 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:05.295990 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4227a0df-53bd-464e-ae8a-5ba55ed4b917 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:05.359766 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-55bec205-5113-4bf4-b538-0b70377ab8a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:05.367969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-55bec205-5113-4bf4-b538-0b70377ab8a5 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1553 microversion: 2.1 time: 0.149155 Jun 10 13:06:05.368620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 64/119] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:05 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1553 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:05.421637 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4227a0df-53bd-464e-ae8a-5ba55ed4b917 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:05.429355 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4227a0df-53bd-464e-ae8a-5ba55ed4b917 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:05.430840 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4227a0df-53bd-464e-ae8a-5ba55ed4b917 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:05.431919 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4227a0df-53bd-464e-ae8a-5ba55ed4b917 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.148479 Jun 10 13:06:05.432470 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 56/120] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:06:05 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:06:06.384198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6c442c5e-7f3d-4313-8756-c272a5c1b62c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:06.384818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6c442c5e-7f3d-4313-8756-c272a5c1b62c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:06.391474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c442c5e-7f3d-4313-8756-c272a5c1b62c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:06.391914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c442c5e-7f3d-4313-8756-c272a5c1b62c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:06.392658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c442c5e-7f3d-4313-8756-c272a5c1b62c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:06.393104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c442c5e-7f3d-4313-8756-c272a5c1b62c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:06.448898 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ce98473f-ef83-4575-8da2-fbb057f5e36c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:06.449509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ce98473f-ef83-4575-8da2-fbb057f5e36c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:06.452918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=63,nova_api:UPDATE=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:06.456403 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ce98473f-ef83-4575-8da2-fbb057f5e36c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:06.456869 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ce98473f-ef83-4575-8da2-fbb057f5e36c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:06.457623 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ce98473f-ef83-4575-8da2-fbb057f5e36c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:06.458063 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ce98473f-ef83-4575-8da2-fbb057f5e36c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:06.518770 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6c442c5e-7f3d-4313-8756-c272a5c1b62c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:06.526584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6c442c5e-7f3d-4313-8756-c272a5c1b62c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1553 microversion: 2.1 time: 0.146162 Jun 10 13:06:06.527163 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 65/121] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:06 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1553 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:06.586118 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ce98473f-ef83-4575-8da2-fbb057f5e36c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:06.593985 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ce98473f-ef83-4575-8da2-fbb057f5e36c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:06.595605 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ce98473f-ef83-4575-8da2-fbb057f5e36c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:06.596719 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ce98473f-ef83-4575-8da2-fbb057f5e36c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.152102 Jun 10 13:06:06.597293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 57/122] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:06:06 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:06:07.543536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a6b6f225-80af-4ba2-9a0d-3444b1cb2b05 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:07.544165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a6b6f225-80af-4ba2-9a0d-3444b1cb2b05 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:07.550294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a6b6f225-80af-4ba2-9a0d-3444b1cb2b05 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:07.550730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a6b6f225-80af-4ba2-9a0d-3444b1cb2b05 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:07.551497 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a6b6f225-80af-4ba2-9a0d-3444b1cb2b05 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:07.551921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a6b6f225-80af-4ba2-9a0d-3444b1cb2b05 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:07.614822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-29c37c29-05e3-406e-86ec-acf856bc738d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:07.615429 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-29c37c29-05e3-406e-86ec-acf856bc738d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:07.622524 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29c37c29-05e3-406e-86ec-acf856bc738d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:07.625216 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29c37c29-05e3-406e-86ec-acf856bc738d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:07.625216 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29c37c29-05e3-406e-86ec-acf856bc738d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:07.625216 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29c37c29-05e3-406e-86ec-acf856bc738d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:07.682159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a6b6f225-80af-4ba2-9a0d-3444b1cb2b05 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:07.691124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a6b6f225-80af-4ba2-9a0d-3444b1cb2b05 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1553 microversion: 2.1 time: 0.151796 Jun 10 13:06:07.691708 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 66/123] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:07 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1553 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:07.797339 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-29c37c29-05e3-406e-86ec-acf856bc738d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:07.809339 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-29c37c29-05e3-406e-86ec-acf856bc738d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:07.810931 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-29c37c29-05e3-406e-86ec-acf856bc738d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:07.812086 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-29c37c29-05e3-406e-86ec-acf856bc738d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1406 microversion: 2.20 time: 0.201490 Jun 10 13:06:07.812996 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 58/124] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:06:07 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1406 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:06:08.711194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6de4ff11-f67a-4513-b5fc-188501320b11 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:08.712657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6de4ff11-f67a-4513-b5fc-188501320b11 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:08.724272 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6de4ff11-f67a-4513-b5fc-188501320b11 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:08.724719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6de4ff11-f67a-4513-b5fc-188501320b11 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:08.729999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6de4ff11-f67a-4513-b5fc-188501320b11 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:08.729999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6de4ff11-f67a-4513-b5fc-188501320b11 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:08.830958 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e9ba4690-e019-432c-a8b2-6da64961d6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:08.831562 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e9ba4690-e019-432c-a8b2-6da64961d6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:08.843347 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9ba4690-e019-432c-a8b2-6da64961d6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:08.843793 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9ba4690-e019-432c-a8b2-6da64961d6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:08.847544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9ba4690-e019-432c-a8b2-6da64961d6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:08.848321 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9ba4690-e019-432c-a8b2-6da64961d6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:08.954101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6de4ff11-f67a-4513-b5fc-188501320b11 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:08.963140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6de4ff11-f67a-4513-b5fc-188501320b11 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1553 microversion: 2.1 time: 0.256482 Jun 10 13:06:08.965079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 67/125] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:08 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1553 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:09.223883 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e9ba4690-e019-432c-a8b2-6da64961d6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:09.226831 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:09.236196 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e9ba4690-e019-432c-a8b2-6da64961d6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:09.238125 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e9ba4690-e019-432c-a8b2-6da64961d6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:09.239477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e9ba4690-e019-432c-a8b2-6da64961d6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1598 microversion: 2.20 time: 0.412927 Jun 10 13:06:09.240036 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 59/126] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:06:08 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1598 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:06:09.257320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-55c9a054-fc33-49a8-b765-f6b2910d8315 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:09.257807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-55c9a054-fc33-49a8-b765-f6b2910d8315 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:09.264391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55c9a054-fc33-49a8-b765-f6b2910d8315 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:09.264937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55c9a054-fc33-49a8-b765-f6b2910d8315 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:09.265703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55c9a054-fc33-49a8-b765-f6b2910d8315 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:09.266134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55c9a054-fc33-49a8-b765-f6b2910d8315 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:09.357630 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-55c9a054-fc33-49a8-b765-f6b2910d8315 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.104298 Jun 10 13:06:09.358356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 68/127] 10.222.0.25 () {60 vars in 1315 bytes} [Fri Jun 10 13:06:09 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-interface => generated 284 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:06:09.966549 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-c38bfa58-2b8b-4525-90db-4e43515e2c73 req-80549e59-50fc-4ff3-914a-639aba4b2010 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1", "name": "network-vif-deleted", "tag": "9e2b9907-e960-46c3-ac8f-00608c9e9b15"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:09.975440 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-c38bfa58-2b8b-4525-90db-4e43515e2c73 req-80549e59-50fc-4ff3-914a-639aba4b2010 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:09.976373 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-c38bfa58-2b8b-4525-90db-4e43515e2c73 req-80549e59-50fc-4ff3-914a-639aba4b2010 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:09.981796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ee78beac-0160-4c20-922c-16f40b550555 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:09.982650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ee78beac-0160-4c20-922c-16f40b550555 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:09.988464 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-c38bfa58-2b8b-4525-90db-4e43515e2c73 req-80549e59-50fc-4ff3-914a-639aba4b2010 service nova] Creating event network-vif-deleted:9e2b9907-e960-46c3-ac8f-00608c9e9b15 for instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 on hv-cinder-maste Jun 10 13:06:09.989095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee78beac-0160-4c20-922c-16f40b550555 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:09.989606 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee78beac-0160-4c20-922c-16f40b550555 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:09.990419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee78beac-0160-4c20-922c-16f40b550555 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:09.990845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee78beac-0160-4c20-922c-16f40b550555 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:09.994387 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-c38bfa58-2b8b-4525-90db-4e43515e2c73 req-80549e59-50fc-4ff3-914a-639aba4b2010 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033208 Jun 10 13:06:09.994990 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 60/128] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:06:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:06:10.052151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ee78beac-0160-4c20-922c-16f40b550555 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:10.060755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ee78beac-0160-4c20-922c-16f40b550555 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1511 microversion: 2.1 time: 0.083135 Jun 10 13:06:10.061340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 69/129] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:09 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1511 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:11.079016 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f8b46044-89e3-40c2-8059-f4216e3bd985 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:11.079922 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f8b46044-89e3-40c2-8059-f4216e3bd985 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:11.087148 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f8b46044-89e3-40c2-8059-f4216e3bd985 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:11.087596 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f8b46044-89e3-40c2-8059-f4216e3bd985 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:11.088360 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f8b46044-89e3-40c2-8059-f4216e3bd985 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:11.088842 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f8b46044-89e3-40c2-8059-f4216e3bd985 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:11.156553 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f8b46044-89e3-40c2-8059-f4216e3bd985 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:11.164254 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f8b46044-89e3-40c2-8059-f4216e3bd985 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1511 microversion: 2.1 time: 0.090473 Jun 10 13:06:11.164848 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 61/130] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:11 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1511 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:12.008623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-dde7b8e1-2f34-444a-9d2d-31600f7429a3 req-8ada7efa-cc38-4683-b750-5326712d0eb1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "db56898a-dff9-4cad-bf8d-f84b53247fc3", "tag": "31e1a9d0-3f1e-4221-a431-639f0e4a6ff4"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:12.017277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-dde7b8e1-2f34-444a-9d2d-31600f7429a3 req-8ada7efa-cc38-4683-b750-5326712d0eb1 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:12.025667 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-dde7b8e1-2f34-444a-9d2d-31600f7429a3 req-8ada7efa-cc38-4683-b750-5326712d0eb1 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:12.029474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-dde7b8e1-2f34-444a-9d2d-31600f7429a3 req-8ada7efa-cc38-4683-b750-5326712d0eb1 service nova] Creating event network-changed:31e1a9d0-3f1e-4221-a431-639f0e4a6ff4 for instance db56898a-dff9-4cad-bf8d-f84b53247fc3 on hv-cinder-maste Jun 10 13:06:12.034771 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-dde7b8e1-2f34-444a-9d2d-31600f7429a3 req-8ada7efa-cc38-4683-b750-5326712d0eb1 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030131 Jun 10 13:06:12.035365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 70/131] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:06:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:06:12.182003 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5132d813-f970-4a63-864d-02cc932cab80 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:12.182640 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5132d813-f970-4a63-864d-02cc932cab80 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:12.190202 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5132d813-f970-4a63-864d-02cc932cab80 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:12.190646 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5132d813-f970-4a63-864d-02cc932cab80 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:12.191399 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5132d813-f970-4a63-864d-02cc932cab80 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:12.194070 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5132d813-f970-4a63-864d-02cc932cab80 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:12.255231 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5132d813-f970-4a63-864d-02cc932cab80 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:12.265929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5132d813-f970-4a63-864d-02cc932cab80 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1511 microversion: 2.1 time: 0.087751 Jun 10 13:06:12.267235 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 62/132] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:12 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1511 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:13.282329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9638124d-c076-4be0-8219-b27a8832f517 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:13.282953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9638124d-c076-4be0-8219-b27a8832f517 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:13.289237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9638124d-c076-4be0-8219-b27a8832f517 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:13.289681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9638124d-c076-4be0-8219-b27a8832f517 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:13.290469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9638124d-c076-4be0-8219-b27a8832f517 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:13.290904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9638124d-c076-4be0-8219-b27a8832f517 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:13.357332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9638124d-c076-4be0-8219-b27a8832f517 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:13.365827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9638124d-c076-4be0-8219-b27a8832f517 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1345 microversion: 2.1 time: 0.087690 Jun 10 13:06:13.366426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 71/133] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:13 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1345 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:14.382846 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-fe1ed9d2-48ad-4335-aaeb-3668fe82c218 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:14.383474 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-fe1ed9d2-48ad-4335-aaeb-3668fe82c218 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:14.390564 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fe1ed9d2-48ad-4335-aaeb-3668fe82c218 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:14.391019 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fe1ed9d2-48ad-4335-aaeb-3668fe82c218 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:14.391794 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fe1ed9d2-48ad-4335-aaeb-3668fe82c218 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:14.392259 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fe1ed9d2-48ad-4335-aaeb-3668fe82c218 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:14.460520 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-fe1ed9d2-48ad-4335-aaeb-3668fe82c218 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:14.469035 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-fe1ed9d2-48ad-4335-aaeb-3668fe82c218 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1345 microversion: 2.1 time: 0.089945 Jun 10 13:06:14.469605 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 63/134] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:14 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1345 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:15.486019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0cc77fd7-e41b-4b2b-ba00-806ecc151977 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:15.486874 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0cc77fd7-e41b-4b2b-ba00-806ecc151977 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:15.494424 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0cc77fd7-e41b-4b2b-ba00-806ecc151977 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:15.494861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0cc77fd7-e41b-4b2b-ba00-806ecc151977 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:15.496120 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0cc77fd7-e41b-4b2b-ba00-806ecc151977 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:15.496542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0cc77fd7-e41b-4b2b-ba00-806ecc151977 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:15.558005 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0cc77fd7-e41b-4b2b-ba00-806ecc151977 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:15.566660 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0cc77fd7-e41b-4b2b-ba00-806ecc151977 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1345 microversion: 2.1 time: 0.084825 Jun 10 13:06:15.567584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 72/135] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:15 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1345 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:16.584071 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c6c3273c-2f52-48a0-91c9-3c7f903d2836 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:16.584681 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c6c3273c-2f52-48a0-91c9-3c7f903d2836 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:16.591306 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c6c3273c-2f52-48a0-91c9-3c7f903d2836 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:16.591766 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c6c3273c-2f52-48a0-91c9-3c7f903d2836 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:16.592561 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c6c3273c-2f52-48a0-91c9-3c7f903d2836 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:16.593027 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c6c3273c-2f52-48a0-91c9-3c7f903d2836 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:16.624808 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c6c3273c-2f52-48a0-91c9-3c7f903d2836 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:16.625110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c6c3273c-2f52-48a0-91c9-3c7f903d2836 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:16.625567 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-c6c3273c-2f52-48a0-91c9-3c7f903d2836 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lazy-loading 'fault' on Instance uuid 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:06:16.672749 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c6c3273c-2f52-48a0-91c9-3c7f903d2836 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43c31865d2d744d6bb809a7048d2c575', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '72affe24c05f4972a9a30883d2cbc85f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:16.680719 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c6c3273c-2f52-48a0-91c9-3c7f903d2836 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 200 len: 1350 microversion: 2.1 time: 0.100389 Jun 10 13:06:16.681535 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 64/136] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:16 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 1350 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:17.705887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bcc832d3-adef-4186-b968-9ab5e9bda596 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:17.707758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bcc832d3-adef-4186-b968-9ab5e9bda596 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:17.714296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bcc832d3-adef-4186-b968-9ab5e9bda596 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:17.714805 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bcc832d3-adef-4186-b968-9ab5e9bda596 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:17.715569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bcc832d3-adef-4186-b968-9ab5e9bda596 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:17.716053 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bcc832d3-adef-4186-b968-9ab5e9bda596 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:17.731665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-bcc832d3-adef-4186-b968-9ab5e9bda596 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] HTTP exception thrown: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bcc832d3-adef-4186-b968-9ab5e9bda596 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Request method failure captured: Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 HTTP/1.1 Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0F-J13KeDL-DDYBrLe3wOP7qgUfb6jUCU1bz0BJG4iEXfc20YRT_98Xq0stSnXw_6VT2R53X09ewJGrhSf5w_PEJsabG5p5cmqFPeZxnNgUY8ekqYuyf8BBQOcQ9JhfVCmJvXVOPhP3Vlm87YxEdfM8mkEkj6hHf93ie5kWxjlnM-4 Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 72affe24c05f4972a9a30883d2cbc85f Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-DeleteServersTestJSON-846215832 Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:06:17.736814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:06:17.738365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/72affe24c05f4972a9a30883d2cbc85f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/72affe24c05f4972a9a30883d2cbc85f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_72affe24c05f4972a9a30883d2cbc85f", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/72affe24c05f4972a9a30883d2cbc85f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:06:17.738365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-DeleteServersTestJSON-846215832 Jun 10 13:06:17.738365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 72affe24c05f4972a9a30883d2cbc85f Jun 10 13:06:17.738365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-DeleteServersTestJSON-846215832 Jun 10 13:06:17.738365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-DeleteServersTestJSON-846215832-project Jun 10 13:06:17.738365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:06:17.738365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:06:17.738365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 43c31865d2d744d6bb809a7048d2c575 Jun 10 13:06:17.738365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-DeleteServersTestJSON-846215832-project Jun 10 13:06:17.738365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:06:17.738365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:17.738365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1'} Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:06:17.739098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:17.740244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:17.741282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bcc832d3-adef-4186-b968-9ab5e9bda596 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Returning 404 to user: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:06:17.741282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bcc832d3-adef-4186-b968-9ab5e9bda596 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 404 len: 111 microversion: 2.1 time: 0.042295 Jun 10 13:06:17.741282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 73/137] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:17 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:06:18.799969 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c2d646a5-dc2c-4fff-bd54-c30ff01247c6 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:18.800644 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c2d646a5-dc2c-4fff-bd54-c30ff01247c6 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:18.807411 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c2d646a5-dc2c-4fff-bd54-c30ff01247c6 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:18.807859 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c2d646a5-dc2c-4fff-bd54-c30ff01247c6 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:18.808656 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c2d646a5-dc2c-4fff-bd54-c30ff01247c6 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:18.809122 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c2d646a5-dc2c-4fff-bd54-c30ff01247c6 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:18.821980 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.wsgi [None req-c2d646a5-dc2c-4fff-bd54-c30ff01247c6 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] HTTP exception thrown: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c2d646a5-dc2c-4fff-bd54-c30ff01247c6 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Request method failure captured: Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: request: GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1/os-volume_attachments HTTP/1.1 Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept: application/json Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept-Encoding: identity Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Connection: close Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Length: 0 Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Type: application/json Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Host: 10.222.0.25 Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Openstack-System-Scope: None Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: User-Agent: python-urllib3/1.26.9 Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Auth-Token: gAAAAABio0F-J13KeDL-DDYBrLe3wOP7qgUfb6jUCU1bz0BJG4iEXfc20YRT_98Xq0stSnXw_6VT2R53X09ewJGrhSf5w_PEJsabG5p5cmqFPeZxnNgUY8ekqYuyf8BBQOcQ9JhfVCmJvXVOPhP3Vlm87YxEdfM8mkEkj6hHf93ie5kWxjlnM-4 Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Id: None Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Name: None Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Identity-Status: Confirmed Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Is-Admin-Project: True Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Id: default Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Name: Default Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Id: 72affe24c05f4972a9a30883d2cbc85f Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Name: tempest-DeleteServersTestJSON-846215832 Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Role: reader,member Jun 10 13:06:18.826739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Roles: reader,member Jun 10 13:06:18.830854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/72affe24c05f4972a9a30883d2cbc85f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/72affe24c05f4972a9a30883d2cbc85f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_72affe24c05f4972a9a30883d2cbc85f", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/72affe24c05f4972a9a30883d2cbc85f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:06:18.830854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant: tempest-DeleteServersTestJSON-846215832 Jun 10 13:06:18.830854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Id: 72affe24c05f4972a9a30883d2cbc85f Jun 10 13:06:18.830854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Name: tempest-DeleteServersTestJSON-846215832 Jun 10 13:06:18.830854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User: tempest-DeleteServersTestJSON-846215832-project Jun 10 13:06:18.830854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Id: default Jun 10 13:06:18.830854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Name: Default Jun 10 13:06:18.830854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Id: 43c31865d2d744d6bb809a7048d2c575 Jun 10 13:06:18.830854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Name: tempest-DeleteServersTestJSON-846215832-project Jun 10 13:06:18.830854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: method: > Jun 10 13:06:18.830854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: exception: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:18.830854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: action_args: {'server_id': '42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1'} Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:06:18.831715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:18.832982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:06:18.834240 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c2d646a5-dc2c-4fff-bd54-c30ff01247c6 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Returning 404 to user: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. {{(pid=118093) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:06:18.834240 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c2d646a5-dc2c-4fff-bd54-c30ff01247c6 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.032293 Jun 10 13:06:18.834240 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 65/138] 10.222.0.25 () {58 vars in 1301 bytes} [Fri Jun 10 13:06:18 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1/os-volume_attachments => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:06:23.736593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-476704131", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452"}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:23.828222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.network.neutron [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] validate_networks() for None {{(pid=118092) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:06:24.137837 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Flavor limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:06:24.138325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Image limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:06:24.138798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Flavor pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:06:24.139276 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Image pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:06:24.139693 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:06:24.149282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:24.149732 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:24.150531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:24.150990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:24.173382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Getting quotas for project d1658a9d6de94030abaf2e7f5c82fc75. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:06:24.177372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Getting quotas for user 07363a81c7e844468ae3e2c203711d78 and project d1658a9d6de94030abaf2e7f5c82fc75. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:06:24.190051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Going to run 1 instances... {{(pid=118092) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:06:24.192720 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118092) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:06:24.337916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b50170ca-5bb3-40bc-8b98-add3b0832400 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.605944 Jun 10 13:06:24.344085 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 74/139] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:06:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 613 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jun 10 13:06:24.349861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-76f5db0a-320a-4451-b823-ca055d512dc7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:24.350459 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-76f5db0a-320a-4451-b823-ca055d512dc7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:24.418992 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-76f5db0a-320a-4451-b823-ca055d512dc7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:24.431536 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-76f5db0a-320a-4451-b823-ca055d512dc7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1214 microversion: 2.1 time: 0.085870 Jun 10 13:06:24.433707 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 66/140] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:24 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1214 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:24.649581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-48a30132-895d-4d17-9e4f-9398f595581c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:24.650151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-48a30132-895d-4d17-9e4f-9398f595581c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:24.656897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48a30132-895d-4d17-9e4f-9398f595581c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:24.657407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48a30132-895d-4d17-9e4f-9398f595581c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:24.658177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48a30132-895d-4d17-9e4f-9398f595581c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:24.658598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48a30132-895d-4d17-9e4f-9398f595581c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:24.672667 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-48a30132-895d-4d17-9e4f-9398f595581c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] HTTP exception thrown: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-48a30132-895d-4d17-9e4f-9398f595581c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Request method failure captured: Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 HTTP/1.1 Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0F-J13KeDL-DDYBrLe3wOP7qgUfb6jUCU1bz0BJG4iEXfc20YRT_98Xq0stSnXw_6VT2R53X09ewJGrhSf5w_PEJsabG5p5cmqFPeZxnNgUY8ekqYuyf8BBQOcQ9JhfVCmJvXVOPhP3Vlm87YxEdfM8mkEkj6hHf93ie5kWxjlnM-4 Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 72affe24c05f4972a9a30883d2cbc85f Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-DeleteServersTestJSON-846215832 Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:06:24.674346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:06:24.675850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/72affe24c05f4972a9a30883d2cbc85f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/72affe24c05f4972a9a30883d2cbc85f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_72affe24c05f4972a9a30883d2cbc85f", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/72affe24c05f4972a9a30883d2cbc85f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:06:24.675850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-DeleteServersTestJSON-846215832 Jun 10 13:06:24.675850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 72affe24c05f4972a9a30883d2cbc85f Jun 10 13:06:24.675850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-DeleteServersTestJSON-846215832 Jun 10 13:06:24.675850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-DeleteServersTestJSON-846215832-project Jun 10 13:06:24.675850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:06:24.675850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:06:24.675850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 43c31865d2d744d6bb809a7048d2c575 Jun 10 13:06:24.675850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-DeleteServersTestJSON-846215832-project Jun 10 13:06:24.675850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:06:24.675850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:24.675850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1'} Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:06:24.676664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:24.678001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:24.679364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-48a30132-895d-4d17-9e4f-9398f595581c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Returning 404 to user: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:06:24.679364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-48a30132-895d-4d17-9e4f-9398f595581c tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "DELETE /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 404 len: 111 microversion: 2.1 time: 0.031634 Jun 10 13:06:24.679364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 75/141] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:06:24 2022] DELETE /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:06:24.686833 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-dd3f9885-a2a6-4800-b5fa-8ba00ed67496 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:24.687448 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-dd3f9885-a2a6-4800-b5fa-8ba00ed67496 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] [instance: 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:24.693959 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dd3f9885-a2a6-4800-b5fa-8ba00ed67496 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:24.694416 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dd3f9885-a2a6-4800-b5fa-8ba00ed67496 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:24.695192 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dd3f9885-a2a6-4800-b5fa-8ba00ed67496 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:24.695633 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dd3f9885-a2a6-4800-b5fa-8ba00ed67496 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:24.712476 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.wsgi [None req-dd3f9885-a2a6-4800-b5fa-8ba00ed67496 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] HTTP exception thrown: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-dd3f9885-a2a6-4800-b5fa-8ba00ed67496 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Request method failure captured: Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: request: GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 HTTP/1.1 Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept: application/json Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept-Encoding: identity Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Connection: close Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Length: 0 Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Type: application/json Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Host: 10.222.0.25 Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Openstack-System-Scope: None Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: User-Agent: python-urllib3/1.26.9 Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Auth-Token: gAAAAABio0F-J13KeDL-DDYBrLe3wOP7qgUfb6jUCU1bz0BJG4iEXfc20YRT_98Xq0stSnXw_6VT2R53X09ewJGrhSf5w_PEJsabG5p5cmqFPeZxnNgUY8ekqYuyf8BBQOcQ9JhfVCmJvXVOPhP3Vlm87YxEdfM8mkEkj6hHf93ie5kWxjlnM-4 Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Id: None Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Name: None Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Identity-Status: Confirmed Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Is-Admin-Project: True Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Id: default Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Name: Default Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Id: 72affe24c05f4972a9a30883d2cbc85f Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Name: tempest-DeleteServersTestJSON-846215832 Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Role: reader,member Jun 10 13:06:24.715152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Roles: reader,member Jun 10 13:06:24.716574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/72affe24c05f4972a9a30883d2cbc85f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/72affe24c05f4972a9a30883d2cbc85f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_72affe24c05f4972a9a30883d2cbc85f", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/72affe24c05f4972a9a30883d2cbc85f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:06:24.716574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant: tempest-DeleteServersTestJSON-846215832 Jun 10 13:06:24.716574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Id: 72affe24c05f4972a9a30883d2cbc85f Jun 10 13:06:24.716574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Name: tempest-DeleteServersTestJSON-846215832 Jun 10 13:06:24.716574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User: tempest-DeleteServersTestJSON-846215832-project Jun 10 13:06:24.716574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Id: default Jun 10 13:06:24.716574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Name: Default Jun 10 13:06:24.716574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Id: 43c31865d2d744d6bb809a7048d2c575 Jun 10 13:06:24.716574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Name: tempest-DeleteServersTestJSON-846215832-project Jun 10 13:06:24.716574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: method: > Jun 10 13:06:24.716574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: exception: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:24.716574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: action_args: {'id': '42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1'} Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:06:24.717477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. Jun 10 13:06:24.718768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:06:24.719933 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-dd3f9885-a2a6-4800-b5fa-8ba00ed67496 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] Returning 404 to user: Instance 42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 could not be found. {{(pid=118093) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:06:24.719933 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-dd3f9885-a2a6-4800-b5fa-8ba00ed67496 tempest-DeleteServersTestJSON-846215832 tempest-DeleteServersTestJSON-846215832-project] 10.222.0.25 "GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1" status: 404 len: 111 microversion: 2.1 time: 0.035326 Jun 10 13:06:24.724312 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 67/142] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:24 2022] GET /compute/v2.1/servers/42f0a4bb-23f9-438e-af4b-7baa2aaaf8c1 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:06:25.455397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6c8d34ba-8682-43fa-9b27-8ea1efce7a45 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:25.455397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6c8d34ba-8682-43fa-9b27-8ea1efce7a45 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:25.512832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6c8d34ba-8682-43fa-9b27-8ea1efce7a45 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:25.520029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6c8d34ba-8682-43fa-9b27-8ea1efce7a45 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1214 microversion: 2.1 time: 0.073357 Jun 10 13:06:25.521547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 76/143] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:25 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1214 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:26.537783 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-13a8155f-4480-4dc6-97f8-308082a2988a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:26.538324 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-13a8155f-4480-4dc6-97f8-308082a2988a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:26.545266 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-13a8155f-4480-4dc6-97f8-308082a2988a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:26.545702 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-13a8155f-4480-4dc6-97f8-308082a2988a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:26.546445 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-13a8155f-4480-4dc6-97f8-308082a2988a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:26.546866 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-13a8155f-4480-4dc6-97f8-308082a2988a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:26.606610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-13a8155f-4480-4dc6-97f8-308082a2988a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:26.614193 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-13a8155f-4480-4dc6-97f8-308082a2988a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1218 microversion: 2.1 time: 0.080327 Jun 10 13:06:26.614757 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 68/144] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:26 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1218 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:27.630556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-61786625-9d6c-4f16-8a9e-3d356c2d3285 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:27.632075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-61786625-9d6c-4f16-8a9e-3d356c2d3285 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:27.639975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61786625-9d6c-4f16-8a9e-3d356c2d3285 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:27.640589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61786625-9d6c-4f16-8a9e-3d356c2d3285 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:27.645169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61786625-9d6c-4f16-8a9e-3d356c2d3285 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:27.645169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61786625-9d6c-4f16-8a9e-3d356c2d3285 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:27.707055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-61786625-9d6c-4f16-8a9e-3d356c2d3285 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:27.718258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-61786625-9d6c-4f16-8a9e-3d356c2d3285 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1210 microversion: 2.1 time: 0.091418 Jun 10 13:06:27.720587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 77/145] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:27 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1210 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:28.735818 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7c7c79da-f555-4d80-bac4-ad8131f5778f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:28.737269 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7c7c79da-f555-4d80-bac4-ad8131f5778f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:28.746736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7c7c79da-f555-4d80-bac4-ad8131f5778f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:28.746736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7c7c79da-f555-4d80-bac4-ad8131f5778f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:28.746736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7c7c79da-f555-4d80-bac4-ad8131f5778f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:28.746736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7c7c79da-f555-4d80-bac4-ad8131f5778f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:28.821584 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7c7c79da-f555-4d80-bac4-ad8131f5778f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:28.830889 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7c7c79da-f555-4d80-bac4-ad8131f5778f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1266 microversion: 2.1 time: 0.099180 Jun 10 13:06:28.831635 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 69/146] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:28 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1266 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:29.847724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-124c2d2a-1d03-42b8-a5c9-a18c7101e8e8 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:29.849719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-124c2d2a-1d03-42b8-a5c9-a18c7101e8e8 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:29.858644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-124c2d2a-1d03-42b8-a5c9-a18c7101e8e8 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:29.859095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-124c2d2a-1d03-42b8-a5c9-a18c7101e8e8 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:29.859866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-124c2d2a-1d03-42b8-a5c9-a18c7101e8e8 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:29.860402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-124c2d2a-1d03-42b8-a5c9-a18c7101e8e8 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:29.937560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-124c2d2a-1d03-42b8-a5c9-a18c7101e8e8 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:29.947415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-124c2d2a-1d03-42b8-a5c9-a18c7101e8e8 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1266 microversion: 2.1 time: 0.103076 Jun 10 13:06:29.948657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 78/147] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:29 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1266 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:30.964318 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-639246ec-4f54-4593-ba85-ac3a0c020681 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:30.965535 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-639246ec-4f54-4593-ba85-ac3a0c020681 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:30.974418 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-639246ec-4f54-4593-ba85-ac3a0c020681 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:30.975131 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-639246ec-4f54-4593-ba85-ac3a0c020681 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:30.975960 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-639246ec-4f54-4593-ba85-ac3a0c020681 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:30.976445 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-639246ec-4f54-4593-ba85-ac3a0c020681 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:31.111629 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-639246ec-4f54-4593-ba85-ac3a0c020681 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:31.134229 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-639246ec-4f54-4593-ba85-ac3a0c020681 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1274 microversion: 2.1 time: 0.173567 Jun 10 13:06:31.136050 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 70/148] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:30 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1274 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:32.154727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a44335da-0f90-47c8-b534-95f103620587 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:32.155252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a44335da-0f90-47c8-b534-95f103620587 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:32.163545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a44335da-0f90-47c8-b534-95f103620587 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:32.167686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a44335da-0f90-47c8-b534-95f103620587 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:32.167686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a44335da-0f90-47c8-b534-95f103620587 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:32.167686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a44335da-0f90-47c8-b534-95f103620587 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:32.254865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a44335da-0f90-47c8-b534-95f103620587 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:32.263331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a44335da-0f90-47c8-b534-95f103620587 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1272 microversion: 2.1 time: 0.113729 Jun 10 13:06:32.263875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 79/149] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:32 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1272 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:33.283744 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9280051e-8f58-4375-ab4c-27e9db23985a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:33.284368 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9280051e-8f58-4375-ab4c-27e9db23985a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:33.294410 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9280051e-8f58-4375-ab4c-27e9db23985a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:33.295661 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9280051e-8f58-4375-ab4c-27e9db23985a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:33.296626 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9280051e-8f58-4375-ab4c-27e9db23985a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:33.297554 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9280051e-8f58-4375-ab4c-27e9db23985a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:33.696643 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9280051e-8f58-4375-ab4c-27e9db23985a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:33.710983 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9280051e-8f58-4375-ab4c-27e9db23985a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1314 microversion: 2.1 time: 0.431678 Jun 10 13:06:33.718776 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 71/150] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:33 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1314 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:34.734346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bbb5f56a-8a39-42ed-b76c-076c972d69a5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:34.734997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bbb5f56a-8a39-42ed-b76c-076c972d69a5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:34.737359 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:34.738067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=49,nova_api:INSERT=3 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:34.744707 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bbb5f56a-8a39-42ed-b76c-076c972d69a5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:34.751256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bbb5f56a-8a39-42ed-b76c-076c972d69a5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:34.751256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bbb5f56a-8a39-42ed-b76c-076c972d69a5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:34.751256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bbb5f56a-8a39-42ed-b76c-076c972d69a5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:34.756362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=76 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:34.850238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-623a9b4b-12ad-4b3d-bf51-5f35e479c164 req-2f636b6f-5033-4744-94a0-99c3ce6d2079 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "37eb1bf3-3476-4006-becd-b9a1ac3603a5", "tag": "6461896a-0fd3-48ee-967d-abff8c615d2a"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:34.870317 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=2 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:34.872429 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-623a9b4b-12ad-4b3d-bf51-5f35e479c164 req-2f636b6f-5033-4744-94a0-99c3ce6d2079 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:34.883726 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-623a9b4b-12ad-4b3d-bf51-5f35e479c164 req-2f636b6f-5033-4744-94a0-99c3ce6d2079 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:34.912351 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-623a9b4b-12ad-4b3d-bf51-5f35e479c164 req-2f636b6f-5033-4744-94a0-99c3ce6d2079 service nova] Creating event network-changed:6461896a-0fd3-48ee-967d-abff8c615d2a for instance 37eb1bf3-3476-4006-becd-b9a1ac3603a5 on hv-cinder-maste Jun 10 13:06:34.940474 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-623a9b4b-12ad-4b3d-bf51-5f35e479c164 req-2f636b6f-5033-4744-94a0-99c3ce6d2079 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.096299 Jun 10 13:06:34.945370 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 72/151] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:06:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:06:35.189947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bbb5f56a-8a39-42ed-b76c-076c972d69a5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:35.211478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bbb5f56a-8a39-42ed-b76c-076c972d69a5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1314 microversion: 2.1 time: 0.478413 Jun 10 13:06:35.212234 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 80/152] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:34 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1314 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:35.657536 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-918507137", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:35.722934 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.network.neutron [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] validate_networks() for [('a8e4f8d9-a40b-43e6-a232-daac66b17f5f', None, None, None, None, None)] {{(pid=118093) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:06:36.228595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4305abef-9c0c-4c79-9374-847bd9a59584 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:36.229101 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Flavor limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:06:36.229469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4305abef-9c0c-4c79-9374-847bd9a59584 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:36.230628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Image limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:06:36.231294 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Flavor pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:06:36.231862 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Image pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:06:36.233009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=8 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:36.233143 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:06:36.238017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4305abef-9c0c-4c79-9374-847bd9a59584 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:36.238660 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4305abef-9c0c-4c79-9374-847bd9a59584 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:36.241011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4305abef-9c0c-4c79-9374-847bd9a59584 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:36.241011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4305abef-9c0c-4c79-9374-847bd9a59584 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:36.244280 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:36.244971 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:36.246645 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:36.247079 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:36.278900 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Getting quotas for project 03423e3482614374afa8c929bd931262. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:06:36.283208 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Getting quotas for user 25f73d0bd4e64057a922d11be46eb04c and project 03423e3482614374afa8c929bd931262. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:06:36.296506 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Going to run 1 instances... {{(pid=118093) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:06:36.298934 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118093) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:06:36.428413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4305abef-9c0c-4c79-9374-847bd9a59584 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:36.441908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4305abef-9c0c-4c79-9374-847bd9a59584 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1314 microversion: 2.1 time: 0.217725 Jun 10 13:06:36.469859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 81/153] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:36 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1314 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:36.485972 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1382b80a-7b7a-43d6-a87a-9739c9c1dfe8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.045733 Jun 10 13:06:36.486662 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 73/154] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:06:35 2022] POST /compute/v2.1/servers => generated 383 bytes in 1048 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jun 10 13:06:36.499793 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3b4bb7be-627a-46c5-ac1a-d2fc1863da74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:36.500101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3b4bb7be-627a-46c5-ac1a-d2fc1863da74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:36.576627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3b4bb7be-627a-46c5-ac1a-d2fc1863da74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:36.586814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3b4bb7be-627a-46c5-ac1a-d2fc1863da74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1221 microversion: 2.1 time: 0.091640 Jun 10 13:06:36.589141 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 82/155] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:36 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1221 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:37.041931 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2120910476", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:37.061893 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=49,nova_api:INSERT=3 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:37.136009 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.network.neutron [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] validate_networks() for [('a8e4f8d9-a40b-43e6-a232-daac66b17f5f', None, None, None, None, None)] {{(pid=118093) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:06:37.459602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0f851075-74e3-489c-babe-9a81f54e19a1 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:37.460212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0f851075-74e3-489c-babe-9a81f54e19a1 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:37.481258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f851075-74e3-489c-babe-9a81f54e19a1 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:37.481258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f851075-74e3-489c-babe-9a81f54e19a1 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:37.481258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f851075-74e3-489c-babe-9a81f54e19a1 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:37.481258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f851075-74e3-489c-babe-9a81f54e19a1 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:37.898832 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Flavor limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:06:37.899548 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Image limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:06:37.900023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Flavor pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:06:37.901580 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Image pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:06:37.905159 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:06:37.924026 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:37.924472 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:37.925299 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:37.927423 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:37.966476 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Getting quotas for project 03423e3482614374afa8c929bd931262. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:06:37.971597 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Getting quotas for user 25f73d0bd4e64057a922d11be46eb04c and project 03423e3482614374afa8c929bd931262. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:06:37.986564 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Going to run 1 instances... {{(pid=118093) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:06:37.988972 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118093) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:06:38.170656 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c0d71eeb-c585-4b12-ab98-8702f7631283 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.123460 Jun 10 13:06:38.172361 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 74/156] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:06:37 2022] POST /compute/v2.1/servers => generated 383 bytes in 1139 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jun 10 13:06:38.175074 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0f851075-74e3-489c-babe-9a81f54e19a1 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:38.188620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d99e4e08-85be-40e8-bdd3-a4901f465ae8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:38.192877 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d99e4e08-85be-40e8-bdd3-a4901f465ae8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:38.199831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0f851075-74e3-489c-babe-9a81f54e19a1 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1314 microversion: 2.1 time: 0.743969 Jun 10 13:06:38.200418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 83/157] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:37 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1314 bytes in 745 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:38.313752 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d99e4e08-85be-40e8-bdd3-a4901f465ae8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:38.352736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d99e4e08-85be-40e8-bdd3-a4901f465ae8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1222 microversion: 2.1 time: 0.175178 Jun 10 13:06:38.356090 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 75/158] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:38 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1222 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:39.219763 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b2267b54-54c6-4970-ac1b-233f70d5a10f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:39.221571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b2267b54-54c6-4970-ac1b-233f70d5a10f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:39.229295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b2267b54-54c6-4970-ac1b-233f70d5a10f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:39.230047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b2267b54-54c6-4970-ac1b-233f70d5a10f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:39.230962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b2267b54-54c6-4970-ac1b-233f70d5a10f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:39.231640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b2267b54-54c6-4970-ac1b-233f70d5a10f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:39.373248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-63d93c4a-f1e6-4daf-bbfb-96fe5db61fff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:39.374213 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-63d93c4a-f1e6-4daf-bbfb-96fe5db61fff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:39.382049 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-63d93c4a-f1e6-4daf-bbfb-96fe5db61fff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:39.382924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-63d93c4a-f1e6-4daf-bbfb-96fe5db61fff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:39.383933 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-63d93c4a-f1e6-4daf-bbfb-96fe5db61fff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:39.384615 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-63d93c4a-f1e6-4daf-bbfb-96fe5db61fff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:39.397641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b2267b54-54c6-4970-ac1b-233f70d5a10f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:39.407554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b2267b54-54c6-4970-ac1b-233f70d5a10f tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1314 microversion: 2.1 time: 0.192722 Jun 10 13:06:39.409055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 84/159] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:39 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1314 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:39.506024 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-63d93c4a-f1e6-4daf-bbfb-96fe5db61fff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:39.518146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-63d93c4a-f1e6-4daf-bbfb-96fe5db61fff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1226 microversion: 2.1 time: 0.149249 Jun 10 13:06:39.520510 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 76/160] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:39 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1226 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:40.426258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8a3d3ae2-e413-46bc-b317-7042c5869dcf tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:40.426883 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8a3d3ae2-e413-46bc-b317-7042c5869dcf tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:40.436289 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a3d3ae2-e413-46bc-b317-7042c5869dcf tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:40.437041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a3d3ae2-e413-46bc-b317-7042c5869dcf tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:40.438680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a3d3ae2-e413-46bc-b317-7042c5869dcf tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:40.439536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a3d3ae2-e413-46bc-b317-7042c5869dcf tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:40.537466 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7ac23289-3140-403a-b11b-396d30cb95f9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:40.539457 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7ac23289-3140-403a-b11b-396d30cb95f9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:40.547473 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7ac23289-3140-403a-b11b-396d30cb95f9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:40.547917 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7ac23289-3140-403a-b11b-396d30cb95f9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:40.548922 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7ac23289-3140-403a-b11b-396d30cb95f9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:40.550292 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7ac23289-3140-403a-b11b-396d30cb95f9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:40.607498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8a3d3ae2-e413-46bc-b317-7042c5869dcf tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:40.621285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8a3d3ae2-e413-46bc-b317-7042c5869dcf tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1314 microversion: 2.1 time: 0.195513 Jun 10 13:06:40.621285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 85/161] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:40 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1314 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:40.658364 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7ac23289-3140-403a-b11b-396d30cb95f9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:40.666430 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7ac23289-3140-403a-b11b-396d30cb95f9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1274 microversion: 2.1 time: 0.133735 Jun 10 13:06:40.666993 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 77/162] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:40 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1274 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:41.565766 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-048f6170-de7f-4ed2-a7e2-cab39213cc41 req-edd6ef65-dd4b-4660-a059-a540554324f5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "37eb1bf3-3476-4006-becd-b9a1ac3603a5", "name": "network-vif-plugged", "status": "completed", "tag": "6461896a-0fd3-48ee-967d-abff8c615d2a"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:41.572065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-048f6170-de7f-4ed2-a7e2-cab39213cc41 req-edd6ef65-dd4b-4660-a059-a540554324f5 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:41.585986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-048f6170-de7f-4ed2-a7e2-cab39213cc41 req-edd6ef65-dd4b-4660-a059-a540554324f5 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:41.597811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-048f6170-de7f-4ed2-a7e2-cab39213cc41 req-edd6ef65-dd4b-4660-a059-a540554324f5 service nova] Creating event network-vif-plugged:6461896a-0fd3-48ee-967d-abff8c615d2a for instance 37eb1bf3-3476-4006-becd-b9a1ac3603a5 on hv-cinder-maste Jun 10 13:06:41.603281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-048f6170-de7f-4ed2-a7e2-cab39213cc41 req-edd6ef65-dd4b-4660-a059-a540554324f5 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044313 Jun 10 13:06:41.604161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 86/163] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:06:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:06:41.642700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-62522856-604b-4bee-aa74-bfc2e5c36a23 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:41.649098 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-62522856-604b-4bee-aa74-bfc2e5c36a23 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:41.664002 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62522856-604b-4bee-aa74-bfc2e5c36a23 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:41.664731 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62522856-604b-4bee-aa74-bfc2e5c36a23 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:41.665884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62522856-604b-4bee-aa74-bfc2e5c36a23 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:41.666029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62522856-604b-4bee-aa74-bfc2e5c36a23 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:41.684433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a82afc9a-9e11-4a1f-8c08-3cc01307b0de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:41.685409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a82afc9a-9e11-4a1f-8c08-3cc01307b0de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:41.725037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a82afc9a-9e11-4a1f-8c08-3cc01307b0de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:41.725878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a82afc9a-9e11-4a1f-8c08-3cc01307b0de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:41.730442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a82afc9a-9e11-4a1f-8c08-3cc01307b0de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:41.731143 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a82afc9a-9e11-4a1f-8c08-3cc01307b0de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:41.914132 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a82afc9a-9e11-4a1f-8c08-3cc01307b0de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:41.926183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a82afc9a-9e11-4a1f-8c08-3cc01307b0de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1280 microversion: 2.1 time: 0.246222 Jun 10 13:06:41.927017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 87/164] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:41 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1280 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:42.582033 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-62522856-604b-4bee-aa74-bfc2e5c36a23 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:42.587567 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=80 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:42.598181 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-62522856-604b-4bee-aa74-bfc2e5c36a23 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1314 microversion: 2.1 time: 0.964362 Jun 10 13:06:42.600599 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 78/165] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:41 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1314 bytes in 967 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:42.884545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-e22805e3-ffdd-40f9-81cd-32e6e1b001ea req-8d792139-f650-4d33-ad82-23f8801a5418 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5d205551-9f23-46f8-b67b-ca5bafd58182", "tag": "9acb878a-5d99-4c1a-b520-5b9f3cdb505f"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:42.899661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-e22805e3-ffdd-40f9-81cd-32e6e1b001ea req-8d792139-f650-4d33-ad82-23f8801a5418 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:42.900649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-e22805e3-ffdd-40f9-81cd-32e6e1b001ea req-8d792139-f650-4d33-ad82-23f8801a5418 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:42.916015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-e22805e3-ffdd-40f9-81cd-32e6e1b001ea req-8d792139-f650-4d33-ad82-23f8801a5418 service nova] Creating event network-changed:9acb878a-5d99-4c1a-b520-5b9f3cdb505f for instance 5d205551-9f23-46f8-b67b-ca5bafd58182 on hv-cinder-maste Jun 10 13:06:42.921709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-e22805e3-ffdd-40f9-81cd-32e6e1b001ea req-8d792139-f650-4d33-ad82-23f8801a5418 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043274 Jun 10 13:06:42.922675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 88/166] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:06:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:06:42.944927 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9263164e-b4e3-4086-85d0-24b254204e06 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:42.945576 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9263164e-b4e3-4086-85d0-24b254204e06 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:42.954012 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9263164e-b4e3-4086-85d0-24b254204e06 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:42.954718 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9263164e-b4e3-4086-85d0-24b254204e06 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:42.955728 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9263164e-b4e3-4086-85d0-24b254204e06 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:42.956330 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9263164e-b4e3-4086-85d0-24b254204e06 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:43.247534 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9263164e-b4e3-4086-85d0-24b254204e06 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:43.257802 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9263164e-b4e3-4086-85d0-24b254204e06 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1322 microversion: 2.1 time: 0.318223 Jun 10 13:06:43.259955 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 79/167] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:42 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1322 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:43.619106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a90898e9-3c49-4cce-956a-9b57de5fee8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:43.619106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a90898e9-3c49-4cce-956a-9b57de5fee8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:43.624479 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a90898e9-3c49-4cce-956a-9b57de5fee8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:43.624980 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a90898e9-3c49-4cce-956a-9b57de5fee8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:43.625749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a90898e9-3c49-4cce-956a-9b57de5fee8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:43.626165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a90898e9-3c49-4cce-956a-9b57de5fee8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:43.780551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a90898e9-3c49-4cce-956a-9b57de5fee8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:43.789676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a90898e9-3c49-4cce-956a-9b57de5fee8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1314 microversion: 2.1 time: 0.176420 Jun 10 13:06:43.797765 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 89/168] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:43 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1314 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:44.275983 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5c33bf4f-d65c-4710-91d8-209196fa18c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:44.277706 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5c33bf4f-d65c-4710-91d8-209196fa18c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:44.286372 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5c33bf4f-d65c-4710-91d8-209196fa18c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:44.287122 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5c33bf4f-d65c-4710-91d8-209196fa18c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:44.288404 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5c33bf4f-d65c-4710-91d8-209196fa18c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:44.289062 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5c33bf4f-d65c-4710-91d8-209196fa18c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:44.619047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5c33bf4f-d65c-4710-91d8-209196fa18c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:44.626902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5c33bf4f-d65c-4710-91d8-209196fa18c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1322 microversion: 2.1 time: 0.356725 Jun 10 13:06:44.629859 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 80/169] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:44 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1322 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:44.807415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a59793f2-3ce6-4039-aa51-aafffac1e3e7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:44.808174 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a59793f2-3ce6-4039-aa51-aafffac1e3e7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:44.816417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a59793f2-3ce6-4039-aa51-aafffac1e3e7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:44.817143 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a59793f2-3ce6-4039-aa51-aafffac1e3e7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:44.818095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a59793f2-3ce6-4039-aa51-aafffac1e3e7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:44.818873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a59793f2-3ce6-4039-aa51-aafffac1e3e7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:45.058538 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a59793f2-3ce6-4039-aa51-aafffac1e3e7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:45.066334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a59793f2-3ce6-4039-aa51-aafffac1e3e7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1463 microversion: 2.1 time: 0.263419 Jun 10 13:06:45.066978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 90/170] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:44 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1463 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:45.195403 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-759e6870-9014-44ea-b579-533a52327b35 req-de233337-9299-494b-9c66-9fbaed672a84 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6ddb4431-e631-4edd-af56-46cbe0241ca8", "tag": "ec274b26-171c-4c68-9350-cbe87c6ec7bc"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:45.209783 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-759e6870-9014-44ea-b579-533a52327b35 req-de233337-9299-494b-9c66-9fbaed672a84 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:45.210233 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-759e6870-9014-44ea-b579-533a52327b35 req-de233337-9299-494b-9c66-9fbaed672a84 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:45.226391 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-759e6870-9014-44ea-b579-533a52327b35 req-de233337-9299-494b-9c66-9fbaed672a84 service nova] Creating event network-changed:ec274b26-171c-4c68-9350-cbe87c6ec7bc for instance 6ddb4431-e631-4edd-af56-46cbe0241ca8 on hv-cinder-maste Jun 10 13:06:45.241131 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-759e6870-9014-44ea-b579-533a52327b35 req-de233337-9299-494b-9c66-9fbaed672a84 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049183 Jun 10 13:06:45.244493 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 81/171] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:06:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:06:45.426534 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-575c947a-f0d6-4f9e-a912-bdb683019cc7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:45.427027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-575c947a-f0d6-4f9e-a912-bdb683019cc7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:45.433462 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-575c947a-f0d6-4f9e-a912-bdb683019cc7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:45.433907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-575c947a-f0d6-4f9e-a912-bdb683019cc7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:45.434659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-575c947a-f0d6-4f9e-a912-bdb683019cc7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:45.435088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-575c947a-f0d6-4f9e-a912-bdb683019cc7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:45.469916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-575c947a-f0d6-4f9e-a912-bdb683019cc7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:06:45.646913 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3bcd7740-4f42-47a8-99cd-63741bf8ffed tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:45.647613 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3bcd7740-4f42-47a8-99cd-63741bf8ffed tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:45.654151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3bcd7740-4f42-47a8-99cd-63741bf8ffed tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:45.654672 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3bcd7740-4f42-47a8-99cd-63741bf8ffed tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:45.655682 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3bcd7740-4f42-47a8-99cd-63741bf8ffed tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:45.656183 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3bcd7740-4f42-47a8-99cd-63741bf8ffed tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:45.802492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3bcd7740-4f42-47a8-99cd-63741bf8ffed tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:45.810912 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3bcd7740-4f42-47a8-99cd-63741bf8ffed tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1322 microversion: 2.1 time: 0.168615 Jun 10 13:06:45.811539 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 82/172] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:45 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1322 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:45.875459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-575c947a-f0d6-4f9e-a912-bdb683019cc7 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "DELETE /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 204 len: 0 microversion: 2.1 time: 0.450756 Jun 10 13:06:45.876292 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 91/173] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:06:45 2022] DELETE /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jun 10 13:06:45.884620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e4eb4825-45b1-4ccf-a05f-6f33aa8809b4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:45.885364 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e4eb4825-45b1-4ccf-a05f-6f33aa8809b4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:45.891402 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e4eb4825-45b1-4ccf-a05f-6f33aa8809b4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:45.891867 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e4eb4825-45b1-4ccf-a05f-6f33aa8809b4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:45.892617 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e4eb4825-45b1-4ccf-a05f-6f33aa8809b4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:45.893069 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e4eb4825-45b1-4ccf-a05f-6f33aa8809b4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:46.022411 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e4eb4825-45b1-4ccf-a05f-6f33aa8809b4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:46.030303 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e4eb4825-45b1-4ccf-a05f-6f33aa8809b4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1469 microversion: 2.1 time: 0.149611 Jun 10 13:06:46.030861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 83/174] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:45 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1469 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:46.828274 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-52bce6d7-0244-46ea-83ac-698af4710d30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:46.829036 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-52bce6d7-0244-46ea-83ac-698af4710d30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:46.835606 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52bce6d7-0244-46ea-83ac-698af4710d30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:46.836082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52bce6d7-0244-46ea-83ac-698af4710d30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:46.836872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52bce6d7-0244-46ea-83ac-698af4710d30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:46.837306 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52bce6d7-0244-46ea-83ac-698af4710d30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:46.963974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-52bce6d7-0244-46ea-83ac-698af4710d30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:46.971325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-52bce6d7-0244-46ea-83ac-698af4710d30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1322 microversion: 2.1 time: 0.147248 Jun 10 13:06:46.971880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 92/175] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:46 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1322 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:47.047129 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-66c14352-00db-4367-833e-a6481abf77c5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:47.047735 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-66c14352-00db-4367-833e-a6481abf77c5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:47.054236 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66c14352-00db-4367-833e-a6481abf77c5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:47.054667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66c14352-00db-4367-833e-a6481abf77c5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:47.055427 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66c14352-00db-4367-833e-a6481abf77c5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:47.055859 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66c14352-00db-4367-833e-a6481abf77c5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:47.189337 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-66c14352-00db-4367-833e-a6481abf77c5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:47.196652 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-66c14352-00db-4367-833e-a6481abf77c5 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1469 microversion: 2.1 time: 0.153292 Jun 10 13:06:47.197273 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 84/176] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:47 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1469 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:47.558987 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-2111175027", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "networks": [{"uuid": "2ece6558-11f6-4246-abf0-a953a99b1ee4"}]}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:47.567342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:47.616378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.network.neutron [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] validate_networks() for [('2ece6558-11f6-4246-abf0-a953a99b1ee4', None, None, None, None, None)] {{(pid=118092) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:06:47.987643 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cb692b26-6f7e-4f8d-8738-04760f09382d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:47.989040 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cb692b26-6f7e-4f8d-8738-04760f09382d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:47.995856 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=4 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:47.997935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cb692b26-6f7e-4f8d-8738-04760f09382d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:47.998615 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cb692b26-6f7e-4f8d-8738-04760f09382d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:47.999343 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cb692b26-6f7e-4f8d-8738-04760f09382d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:47.999765 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cb692b26-6f7e-4f8d-8738-04760f09382d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:48.165098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Flavor limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:06:48.165556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Image limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:06:48.166015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Flavor pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:06:48.166422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Image pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:06:48.167389 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:06:48.176462 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:48.176973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:48.178486 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:48.178925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:48.197670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Getting quotas for project ff2616337ae440a89047fd9a50218387. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:06:48.202225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Getting quotas for user 849e36f8653d4d1f9a7fb1df8fb913a9 and project ff2616337ae440a89047fd9a50218387. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:06:48.215409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Going to run 1 instances... {{(pid=118092) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:06:48.221781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118092) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:06:48.237915 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cb692b26-6f7e-4f8d-8738-04760f09382d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:48.246730 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cb692b26-6f7e-4f8d-8738-04760f09382d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1322 microversion: 2.1 time: 0.263128 Jun 10 13:06:48.247860 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 85/177] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:47 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1322 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:48.252494 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7f2ec0c3-b4e3-41c9-a074-27f273e378d4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:48.253125 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7f2ec0c3-b4e3-41c9-a074-27f273e378d4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:48.258967 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7f2ec0c3-b4e3-41c9-a074-27f273e378d4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:48.259379 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7f2ec0c3-b4e3-41c9-a074-27f273e378d4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:48.260180 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7f2ec0c3-b4e3-41c9-a074-27f273e378d4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:48.261171 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7f2ec0c3-b4e3-41c9-a074-27f273e378d4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:48.346740 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=14 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:48.378107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b7fc3259-37d6-4c00-b1f8-4fc747069dad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.822994 Jun 10 13:06:48.378792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 93/178] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:06:47 2022] POST /compute/v2.1/servers => generated 380 bytes in 824 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jun 10 13:06:48.389089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a93cb7c7-d793-403c-99a6-105c896d06e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:48.389699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a93cb7c7-d793-403c-99a6-105c896d06e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:48.452578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a93cb7c7-d793-403c-99a6-105c896d06e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:48.462339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a93cb7c7-d793-403c-99a6-105c896d06e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1219 microversion: 2.1 time: 0.076685 Jun 10 13:06:48.473256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 94/179] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:48 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:48.665131 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7f2ec0c3-b4e3-41c9-a074-27f273e378d4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:48.673046 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7f2ec0c3-b4e3-41c9-a074-27f273e378d4 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1469 microversion: 2.1 time: 0.424039 Jun 10 13:06:48.673614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 86/180] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:48 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1469 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:49.264332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-158dff65-bb35-4327-a7fa-9437af034852 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:49.265151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-158dff65-bb35-4327-a7fa-9437af034852 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:49.272361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-158dff65-bb35-4327-a7fa-9437af034852 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:49.272898 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-158dff65-bb35-4327-a7fa-9437af034852 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:49.273643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-158dff65-bb35-4327-a7fa-9437af034852 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:49.274067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-158dff65-bb35-4327-a7fa-9437af034852 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:49.346307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-31c5eeb1-f874-4c65-8501-a28a77f9642e req-1970024e-b668-414e-af62-a52a864ca9db service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "37eb1bf3-3476-4006-becd-b9a1ac3603a5", "name": "network-vif-deleted", "tag": "6461896a-0fd3-48ee-967d-abff8c615d2a"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:49.362772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-31c5eeb1-f874-4c65-8501-a28a77f9642e req-1970024e-b668-414e-af62-a52a864ca9db service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:49.363217 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-31c5eeb1-f874-4c65-8501-a28a77f9642e req-1970024e-b668-414e-af62-a52a864ca9db service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:49.376859 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-31c5eeb1-f874-4c65-8501-a28a77f9642e req-1970024e-b668-414e-af62-a52a864ca9db service nova] Creating event network-vif-deleted:6461896a-0fd3-48ee-967d-abff8c615d2a for instance 37eb1bf3-3476-4006-becd-b9a1ac3603a5 on hv-cinder-maste Jun 10 13:06:49.382111 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-31c5eeb1-f874-4c65-8501-a28a77f9642e req-1970024e-b668-414e-af62-a52a864ca9db service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040179 Jun 10 13:06:49.390371 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 87/181] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:06:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:06:49.482006 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-85c95ca2-14d8-417d-8b56-b24c114444b3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:49.482935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-85c95ca2-14d8-417d-8b56-b24c114444b3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:49.539440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-158dff65-bb35-4327-a7fa-9437af034852 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:49.549651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-158dff65-bb35-4327-a7fa-9437af034852 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1322 microversion: 2.1 time: 0.288998 Jun 10 13:06:49.550653 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-85c95ca2-14d8-417d-8b56-b24c114444b3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:49.551728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 95/182] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:49 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1322 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:49.558902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-85c95ca2-14d8-417d-8b56-b24c114444b3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1219 microversion: 2.1 time: 0.082418 Jun 10 13:06:49.559420 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 88/183] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:49 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:49.668398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-93ff086b-b4bc-4eea-aefc-ec63d9f0003e req-c0746ed1-273e-4ec7-af42-ca7aafd5a2ff service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5d205551-9f23-46f8-b67b-ca5bafd58182", "name": "network-vif-plugged", "status": "completed", "tag": "9acb878a-5d99-4c1a-b520-5b9f3cdb505f"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:49.678137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-93ff086b-b4bc-4eea-aefc-ec63d9f0003e req-c0746ed1-273e-4ec7-af42-ca7aafd5a2ff service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:49.678640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-93ff086b-b4bc-4eea-aefc-ec63d9f0003e req-c0746ed1-273e-4ec7-af42-ca7aafd5a2ff service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:49.690967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-93ff086b-b4bc-4eea-aefc-ec63d9f0003e req-c0746ed1-273e-4ec7-af42-ca7aafd5a2ff service nova] Creating event network-vif-plugged:9acb878a-5d99-4c1a-b520-5b9f3cdb505f for instance 5d205551-9f23-46f8-b67b-ca5bafd58182 on hv-cinder-maste Jun 10 13:06:49.691344 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9688535e-eb30-40c0-99a0-46eff9e1b619 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:49.691949 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9688535e-eb30-40c0-99a0-46eff9e1b619 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:49.696830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-93ff086b-b4bc-4eea-aefc-ec63d9f0003e req-c0746ed1-273e-4ec7-af42-ca7aafd5a2ff service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033448 Jun 10 13:06:49.697652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 96/184] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:06:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:06:49.701224 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9688535e-eb30-40c0-99a0-46eff9e1b619 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:49.701592 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9688535e-eb30-40c0-99a0-46eff9e1b619 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:49.702360 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9688535e-eb30-40c0-99a0-46eff9e1b619 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:49.702785 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9688535e-eb30-40c0-99a0-46eff9e1b619 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:49.767952 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9688535e-eb30-40c0-99a0-46eff9e1b619 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:49.775526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9688535e-eb30-40c0-99a0-46eff9e1b619 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1427 microversion: 2.1 time: 0.088562 Jun 10 13:06:49.776128 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 89/185] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:49 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:50.568292 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-110825c9-94fb-4279-a100-f1cc6deaedeb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:50.569753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-110825c9-94fb-4279-a100-f1cc6deaedeb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:50.575645 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-26813b3b-1855-4f0b-8433-f833715d1b4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:50.576257 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-26813b3b-1855-4f0b-8433-f833715d1b4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:50.577134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-110825c9-94fb-4279-a100-f1cc6deaedeb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:50.577625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-110825c9-94fb-4279-a100-f1cc6deaedeb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:50.581210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-110825c9-94fb-4279-a100-f1cc6deaedeb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:50.581210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-110825c9-94fb-4279-a100-f1cc6deaedeb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:50.582763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26813b3b-1855-4f0b-8433-f833715d1b4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:50.583215 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26813b3b-1855-4f0b-8433-f833715d1b4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:50.583980 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26813b3b-1855-4f0b-8433-f833715d1b4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:50.584472 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26813b3b-1855-4f0b-8433-f833715d1b4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:50.736453 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-26813b3b-1855-4f0b-8433-f833715d1b4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:50.747010 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-26813b3b-1855-4f0b-8433-f833715d1b4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1223 microversion: 2.1 time: 0.175584 Jun 10 13:06:50.747386 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-110825c9-94fb-4279-a100-f1cc6deaedeb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:50.752503 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 90/186] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:50 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1223 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:50.756229 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-110825c9-94fb-4279-a100-f1cc6deaedeb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1322 microversion: 2.1 time: 0.191917 Jun 10 13:06:50.758496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 97/187] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:50 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1322 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:50.768240 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-be3825c1-d107-4ee2-b446-ee649537d61e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:50.768861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-be3825c1-d107-4ee2-b446-ee649537d61e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:50.777481 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-be3825c1-d107-4ee2-b446-ee649537d61e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:50.777999 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-be3825c1-d107-4ee2-b446-ee649537d61e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:50.778868 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-be3825c1-d107-4ee2-b446-ee649537d61e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:50.779546 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-be3825c1-d107-4ee2-b446-ee649537d61e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:50.792172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b97981d7-6e16-480d-b0f2-77c39c7eea26 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:50.793072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b97981d7-6e16-480d-b0f2-77c39c7eea26 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:50.799521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b97981d7-6e16-480d-b0f2-77c39c7eea26 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:50.800687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b97981d7-6e16-480d-b0f2-77c39c7eea26 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:50.801370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b97981d7-6e16-480d-b0f2-77c39c7eea26 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:50.801790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b97981d7-6e16-480d-b0f2-77c39c7eea26 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:50.804407 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-be3825c1-d107-4ee2-b446-ee649537d61e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/ips" status: 200 len: 146 microversion: 2.1 time: 0.049515 Jun 10 13:06:50.809982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 91/188] 10.222.0.25 () {58 vars in 1247 bytes} [Fri Jun 10 13:06:50 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/ips => generated 146 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:06:50.835446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b97981d7-6e16-480d-b0f2-77c39c7eea26 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:50.835882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b97981d7-6e16-480d-b0f2-77c39c7eea26 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:50.836328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-b97981d7-6e16-480d-b0f2-77c39c7eea26 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lazy-loading 'fault' on Instance uuid 37eb1bf3-3476-4006-becd-b9a1ac3603a5 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:06:50.879207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b97981d7-6e16-480d-b0f2-77c39c7eea26 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:50.887600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b97981d7-6e16-480d-b0f2-77c39c7eea26 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1304 microversion: 2.1 time: 0.099219 Jun 10 13:06:50.888172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 98/189] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:50 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1304 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:51.771043 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-496bfb67-2acb-4b2a-9602-24431e10ca4c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:51.771749 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-496bfb67-2acb-4b2a-9602-24431e10ca4c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:51.784220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c66e3471-34e0-4cf9-be71-569995b28fb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:51.784220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c66e3471-34e0-4cf9-be71-569995b28fb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:51.784735 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-496bfb67-2acb-4b2a-9602-24431e10ca4c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:51.784735 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-496bfb67-2acb-4b2a-9602-24431e10ca4c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:51.784735 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-496bfb67-2acb-4b2a-9602-24431e10ca4c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:51.784735 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-496bfb67-2acb-4b2a-9602-24431e10ca4c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:51.785075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c66e3471-34e0-4cf9-be71-569995b28fb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:51.785075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c66e3471-34e0-4cf9-be71-569995b28fb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:51.785075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c66e3471-34e0-4cf9-be71-569995b28fb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:51.785075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c66e3471-34e0-4cf9-be71-569995b28fb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:51.800110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:51.861280 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-496bfb67-2acb-4b2a-9602-24431e10ca4c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:51.870780 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-496bfb67-2acb-4b2a-9602-24431e10ca4c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1271 microversion: 2.1 time: 0.105555 Jun 10 13:06:51.875788 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 92/190] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:51 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1271 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:51.910044 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-616e8937-ce60-4462-b0a5-f891c263e74a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:51.910044 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-616e8937-ce60-4462-b0a5-f891c263e74a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:51.917797 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-616e8937-ce60-4462-b0a5-f891c263e74a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:51.921534 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-616e8937-ce60-4462-b0a5-f891c263e74a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:51.922448 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-616e8937-ce60-4462-b0a5-f891c263e74a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:51.922948 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-616e8937-ce60-4462-b0a5-f891c263e74a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:51.955127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c66e3471-34e0-4cf9-be71-569995b28fb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:51.962622 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-616e8937-ce60-4462-b0a5-f891c263e74a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:51.963295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c66e3471-34e0-4cf9-be71-569995b28fb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1322 microversion: 2.1 time: 0.191676 Jun 10 13:06:51.963462 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-616e8937-ce60-4462-b0a5-f891c263e74a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:51.963865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 99/191] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:51 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1322 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:51.964380 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-616e8937-ce60-4462-b0a5-f891c263e74a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lazy-loading 'fault' on Instance uuid 37eb1bf3-3476-4006-becd-b9a1ac3603a5 {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:06:52.019415 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-616e8937-ce60-4462-b0a5-f891c263e74a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:52.027943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-616e8937-ce60-4462-b0a5-f891c263e74a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1304 microversion: 2.1 time: 0.126805 Jun 10 13:06:52.028846 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 93/192] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:51 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1304 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:52.892948 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f9da9282-5d5e-48b6-8996-77c02a79acd8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:52.894022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f9da9282-5d5e-48b6-8996-77c02a79acd8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:52.901594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f9da9282-5d5e-48b6-8996-77c02a79acd8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:52.902188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f9da9282-5d5e-48b6-8996-77c02a79acd8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:52.903045 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f9da9282-5d5e-48b6-8996-77c02a79acd8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:52.903576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f9da9282-5d5e-48b6-8996-77c02a79acd8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:52.967104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f9da9282-5d5e-48b6-8996-77c02a79acd8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:52.983135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f9da9282-5d5e-48b6-8996-77c02a79acd8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1271 microversion: 2.1 time: 0.095418 Jun 10 13:06:52.988514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 100/193] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:52 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1271 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:52.988622 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a8e8c84c-5e41-44b1-9b8c-721b723e4fde tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:52.988622 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a8e8c84c-5e41-44b1-9b8c-721b723e4fde tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:52.997758 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a8e8c84c-5e41-44b1-9b8c-721b723e4fde tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:52.998197 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a8e8c84c-5e41-44b1-9b8c-721b723e4fde tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:52.998954 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a8e8c84c-5e41-44b1-9b8c-721b723e4fde tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:52.999379 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a8e8c84c-5e41-44b1-9b8c-721b723e4fde tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:53.046793 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d49344ef-9a0c-485c-9cc0-fe4e5025bedd tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:53.047412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d49344ef-9a0c-485c-9cc0-fe4e5025bedd tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:53.053773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d49344ef-9a0c-485c-9cc0-fe4e5025bedd tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:53.054281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d49344ef-9a0c-485c-9cc0-fe4e5025bedd tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:53.055102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d49344ef-9a0c-485c-9cc0-fe4e5025bedd tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:53.055597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d49344ef-9a0c-485c-9cc0-fe4e5025bedd tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:53.093525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d49344ef-9a0c-485c-9cc0-fe4e5025bedd tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:53.094063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d49344ef-9a0c-485c-9cc0-fe4e5025bedd tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:53.094576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-d49344ef-9a0c-485c-9cc0-fe4e5025bedd tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lazy-loading 'fault' on Instance uuid 37eb1bf3-3476-4006-becd-b9a1ac3603a5 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:06:53.136029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d49344ef-9a0c-485c-9cc0-fe4e5025bedd tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:53.136871 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a8e8c84c-5e41-44b1-9b8c-721b723e4fde tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:53.143569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d49344ef-9a0c-485c-9cc0-fe4e5025bedd tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1304 microversion: 2.1 time: 0.101413 Jun 10 13:06:53.144122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 101/194] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:53 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1304 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:53.144721 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a8e8c84c-5e41-44b1-9b8c-721b723e4fde tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1322 microversion: 2.1 time: 0.161087 Jun 10 13:06:53.145643 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 94/195] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:52 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1322 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:54.001646 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ff59eb6d-9b43-4408-af09-816b5c51d003 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:54.002376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ff59eb6d-9b43-4408-af09-816b5c51d003 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:54.013780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff59eb6d-9b43-4408-af09-816b5c51d003 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:54.013780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff59eb6d-9b43-4408-af09-816b5c51d003 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:54.013780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff59eb6d-9b43-4408-af09-816b5c51d003 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:54.013780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff59eb6d-9b43-4408-af09-816b5c51d003 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:54.081420 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ff59eb6d-9b43-4408-af09-816b5c51d003 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:54.089304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ff59eb6d-9b43-4408-af09-816b5c51d003 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1271 microversion: 2.1 time: 0.093519 Jun 10 13:06:54.093217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 102/196] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:53 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1271 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:54.161966 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-deacce38-7169-4dec-be50-3e68adc17c35 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:54.162586 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c3855622-9bb8-4cc0-bb2e-d34ff8bceb8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:54.163051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-deacce38-7169-4dec-be50-3e68adc17c35 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:54.163316 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c3855622-9bb8-4cc0-bb2e-d34ff8bceb8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:54.180488 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c3855622-9bb8-4cc0-bb2e-d34ff8bceb8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:54.180488 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c3855622-9bb8-4cc0-bb2e-d34ff8bceb8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:54.180488 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c3855622-9bb8-4cc0-bb2e-d34ff8bceb8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:54.180488 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c3855622-9bb8-4cc0-bb2e-d34ff8bceb8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:54.181051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-deacce38-7169-4dec-be50-3e68adc17c35 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:54.181051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-deacce38-7169-4dec-be50-3e68adc17c35 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:54.181051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-deacce38-7169-4dec-be50-3e68adc17c35 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:54.181051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-deacce38-7169-4dec-be50-3e68adc17c35 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:54.220724 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c3855622-9bb8-4cc0-bb2e-d34ff8bceb8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:54.220724 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c3855622-9bb8-4cc0-bb2e-d34ff8bceb8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:54.220724 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-c3855622-9bb8-4cc0-bb2e-d34ff8bceb8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lazy-loading 'fault' on Instance uuid 37eb1bf3-3476-4006-becd-b9a1ac3603a5 {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:06:54.267636 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c3855622-9bb8-4cc0-bb2e-d34ff8bceb8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:54.275194 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c3855622-9bb8-4cc0-bb2e-d34ff8bceb8a tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1304 microversion: 2.1 time: 0.117742 Jun 10 13:06:54.276657 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 95/197] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:54 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1304 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:54.315216 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-deacce38-7169-4dec-be50-3e68adc17c35 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:54.322806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-deacce38-7169-4dec-be50-3e68adc17c35 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1322 microversion: 2.1 time: 0.165160 Jun 10 13:06:54.323535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 103/198] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:54 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1322 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:54.801389 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b4c7f2c3-8299-49a4-9a37-2abd55c06566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:54.801879 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b4c7f2c3-8299-49a4-9a37-2abd55c06566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:54.807780 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b4c7f2c3-8299-49a4-9a37-2abd55c06566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:54.808233 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b4c7f2c3-8299-49a4-9a37-2abd55c06566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:54.809012 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b4c7f2c3-8299-49a4-9a37-2abd55c06566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:54.809444 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b4c7f2c3-8299-49a4-9a37-2abd55c06566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:54.832500 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b4c7f2c3-8299-49a4-9a37-2abd55c06566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/ips" status: 200 len: 151 microversion: 2.20 time: 0.034852 Jun 10 13:06:54.833213 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 96/199] 10.222.0.25 () {60 vars in 1288 bytes} [Fri Jun 10 13:06:54 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/ips => generated 151 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:06:55.106101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e19b6006-6cd3-4ce3-8179-234453beb831 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:55.106714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e19b6006-6cd3-4ce3-8179-234453beb831 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:55.115077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e19b6006-6cd3-4ce3-8179-234453beb831 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:55.115917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e19b6006-6cd3-4ce3-8179-234453beb831 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:55.117343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e19b6006-6cd3-4ce3-8179-234453beb831 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:55.118126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e19b6006-6cd3-4ce3-8179-234453beb831 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:55.181015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e19b6006-6cd3-4ce3-8179-234453beb831 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:55.188455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e19b6006-6cd3-4ce3-8179-234453beb831 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1279 microversion: 2.1 time: 0.086622 Jun 10 13:06:55.189042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 104/200] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:55 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1279 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:55.294518 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4325eb0c-6b81-45f9-bae1-9a67db8a9387 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:55.295791 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4325eb0c-6b81-45f9-bae1-9a67db8a9387 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:55.303801 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4325eb0c-6b81-45f9-bae1-9a67db8a9387 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:55.304874 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4325eb0c-6b81-45f9-bae1-9a67db8a9387 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:55.307565 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4325eb0c-6b81-45f9-bae1-9a67db8a9387 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:55.307565 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4325eb0c-6b81-45f9-bae1-9a67db8a9387 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:55.341634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1ea9c36e-9818-4fe8-88bc-d85efdf6c25d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:55.341828 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4325eb0c-6b81-45f9-bae1-9a67db8a9387 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:55.342091 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4325eb0c-6b81-45f9-bae1-9a67db8a9387 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:55.342554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1ea9c36e-9818-4fe8-88bc-d85efdf6c25d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:55.342719 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-4325eb0c-6b81-45f9-bae1-9a67db8a9387 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lazy-loading 'fault' on Instance uuid 37eb1bf3-3476-4006-becd-b9a1ac3603a5 {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:06:55.344572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:55.351800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ea9c36e-9818-4fe8-88bc-d85efdf6c25d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:55.352291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ea9c36e-9818-4fe8-88bc-d85efdf6c25d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:55.353449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ea9c36e-9818-4fe8-88bc-d85efdf6c25d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:55.354218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ea9c36e-9818-4fe8-88bc-d85efdf6c25d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:55.394828 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4325eb0c-6b81-45f9-bae1-9a67db8a9387 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '07363a81c7e844468ae3e2c203711d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1658a9d6de94030abaf2e7f5c82fc75', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:55.402342 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4325eb0c-6b81-45f9-bae1-9a67db8a9387 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 200 len: 1304 microversion: 2.1 time: 0.113069 Jun 10 13:06:55.402908 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 97/201] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:55 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 1304 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:55.510337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1ea9c36e-9818-4fe8-88bc-d85efdf6c25d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:55.520935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1ea9c36e-9818-4fe8-88bc-d85efdf6c25d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1322 microversion: 2.1 time: 0.182480 Jun 10 13:06:55.520935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 105/202] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:55 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1322 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:55.863988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6916cee8-fc90-440d-b842-edf67ff7b19d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6b8950d0-dd0c-47fe-a7bd-d91571ff68ba"}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:55.866713 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6916cee8-fc90-440d-b842-edf67ff7b19d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:55.874977 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6916cee8-fc90-440d-b842-edf67ff7b19d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:55.875572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6916cee8-fc90-440d-b842-edf67ff7b19d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:55.876489 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6916cee8-fc90-440d-b842-edf67ff7b19d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:55.877527 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6916cee8-fc90-440d-b842-edf67ff7b19d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:56.207154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-625f737c-a791-43e1-b854-e74478efd9d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:56.209066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-625f737c-a791-43e1-b854-e74478efd9d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:56.218068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-625f737c-a791-43e1-b854-e74478efd9d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:56.218481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-625f737c-a791-43e1-b854-e74478efd9d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:56.219660 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-625f737c-a791-43e1-b854-e74478efd9d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:56.220246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-625f737c-a791-43e1-b854-e74478efd9d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:56.308329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-625f737c-a791-43e1-b854-e74478efd9d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:56.317699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-625f737c-a791-43e1-b854-e74478efd9d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1277 microversion: 2.1 time: 0.115385 Jun 10 13:06:56.319242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 106/203] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:56 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1277 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:56.424245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4395b7fa-345b-4351-9a72-a7a9113ab018 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:56.431440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4395b7fa-345b-4351-9a72-a7a9113ab018 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] [instance: 37eb1bf3-3476-4006-becd-b9a1ac3603a5] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:56.440275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4395b7fa-345b-4351-9a72-a7a9113ab018 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:56.441854 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4395b7fa-345b-4351-9a72-a7a9113ab018 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:56.443934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4395b7fa-345b-4351-9a72-a7a9113ab018 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:56.451595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4395b7fa-345b-4351-9a72-a7a9113ab018 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:56.467512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-4395b7fa-345b-4351-9a72-a7a9113ab018 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] HTTP exception thrown: Instance 37eb1bf3-3476-4006-becd-b9a1ac3603a5 could not be found. Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4395b7fa-345b-4351-9a72-a7a9113ab018 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Request method failure captured: Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 HTTP/1.1 Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0HMw9dNbLPEbnIii2-DU_9WbYpHG-MTCxAOArFPS-fqKFN-rW20EVIdxR7X-47hx0yOAcFsg_R-JPXwSPLhgxrm3ZaPBCpi8jYV_lNIDs24FPrCYOgkASPQhv5SRC6OBS3RRtldYC3iZGbyDtf13D27EbNvoDCyeNyL54Lu8lqqAh4 Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: d1658a9d6de94030abaf2e7f5c82fc75 Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesNegativeTest-313611012 Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:06:56.469692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:06:56.471251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/d1658a9d6de94030abaf2e7f5c82fc75"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/d1658a9d6de94030abaf2e7f5c82fc75"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_d1658a9d6de94030abaf2e7f5c82fc75", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/d1658a9d6de94030abaf2e7f5c82fc75"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:06:56.471251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesNegativeTest-313611012 Jun 10 13:06:56.471251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: d1658a9d6de94030abaf2e7f5c82fc75 Jun 10 13:06:56.471251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesNegativeTest-313611012 Jun 10 13:06:56.471251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesNegativeTest-313611012-project Jun 10 13:06:56.471251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:06:56.471251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:06:56.471251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 07363a81c7e844468ae3e2c203711d78 Jun 10 13:06:56.471251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesNegativeTest-313611012-project Jun 10 13:06:56.471251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:06:56.471251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 37eb1bf3-3476-4006-becd-b9a1ac3603a5 could not be found. Jun 10 13:06:56.471251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '37eb1bf3-3476-4006-becd-b9a1ac3603a5'} Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 37eb1bf3-3476-4006-becd-b9a1ac3603a5 could not be found. Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:06:56.472113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 37eb1bf3-3476-4006-becd-b9a1ac3603a5 could not be found. Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 37eb1bf3-3476-4006-becd-b9a1ac3603a5 could not be found. Jun 10 13:06:56.473533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:06:56.474745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4395b7fa-345b-4351-9a72-a7a9113ab018 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] Returning 404 to user: Instance 37eb1bf3-3476-4006-becd-b9a1ac3603a5 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:06:56.474745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4395b7fa-345b-4351-9a72-a7a9113ab018 tempest-VolumesNegativeTest-313611012 tempest-VolumesNegativeTest-313611012-project] 10.222.0.25 "GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5" status: 404 len: 111 microversion: 2.1 time: 0.050637 Jun 10 13:06:56.474745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 107/204] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:56 2022] GET /compute/v2.1/servers/37eb1bf3-3476-4006-becd-b9a1ac3603a5 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:06:56.536984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-581df42b-d8fc-48b0-b3fb-a8c7d3fec4c6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:56.537591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-581df42b-d8fc-48b0-b3fb-a8c7d3fec4c6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:56.546158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-581df42b-d8fc-48b0-b3fb-a8c7d3fec4c6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:56.549430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-581df42b-d8fc-48b0-b3fb-a8c7d3fec4c6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:56.550384 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-581df42b-d8fc-48b0-b3fb-a8c7d3fec4c6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:56.550984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-581df42b-d8fc-48b0-b3fb-a8c7d3fec4c6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:56.895345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-581df42b-d8fc-48b0-b3fb-a8c7d3fec4c6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:56.910570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-581df42b-d8fc-48b0-b3fb-a8c7d3fec4c6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1322 microversion: 2.1 time: 0.377664 Jun 10 13:06:56.913521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 108/205] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:56 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1322 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:56.923812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-fe3f023c-3c4c-4602-b20d-02ae6e4cef8e req-3fa1ef9f-1d12-4957-8e61-138ca3d72106 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6ddb4431-e631-4edd-af56-46cbe0241ca8", "name": "network-vif-plugged", "status": "completed", "tag": "ec274b26-171c-4c68-9350-cbe87c6ec7bc"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:56.940435 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-fe3f023c-3c4c-4602-b20d-02ae6e4cef8e req-3fa1ef9f-1d12-4957-8e61-138ca3d72106 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:56.941224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-fe3f023c-3c4c-4602-b20d-02ae6e4cef8e req-3fa1ef9f-1d12-4957-8e61-138ca3d72106 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:56.960332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-fe3f023c-3c4c-4602-b20d-02ae6e4cef8e req-3fa1ef9f-1d12-4957-8e61-138ca3d72106 service nova] Creating event network-vif-plugged:ec274b26-171c-4c68-9350-cbe87c6ec7bc for instance 6ddb4431-e631-4edd-af56-46cbe0241ca8 on hv-cinder-maste Jun 10 13:06:56.965940 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-fe3f023c-3c4c-4602-b20d-02ae6e4cef8e req-3fa1ef9f-1d12-4957-8e61-138ca3d72106 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050784 Jun 10 13:06:56.966927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 109/206] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:06:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:06:56.998716 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6916cee8-fc90-440d-b842-edf67ff7b19d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "POST /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.138457 Jun 10 13:06:57.000978 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 98/207] 10.222.0.25 () {60 vars in 1322 bytes} [Fri Jun 10 13:06:55 2022] POST /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/os-volume_attachments => generated 194 bytes in 1141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:06:57.344229 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ddf55bb6-feb0-450d-a212-48ed041d77d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:57.345462 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ddf55bb6-feb0-450d-a212-48ed041d77d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:57.365450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ddf55bb6-feb0-450d-a212-48ed041d77d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:57.365906 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ddf55bb6-feb0-450d-a212-48ed041d77d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:57.366893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ddf55bb6-feb0-450d-a212-48ed041d77d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:57.367683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ddf55bb6-feb0-450d-a212-48ed041d77d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:57.492431 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d67ba037-b0f2-4445-97ad-fd63463ecfe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:57.493444 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d67ba037-b0f2-4445-97ad-fd63463ecfe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:57.505933 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d67ba037-b0f2-4445-97ad-fd63463ecfe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:57.506875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d67ba037-b0f2-4445-97ad-fd63463ecfe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:57.509210 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d67ba037-b0f2-4445-97ad-fd63463ecfe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:57.510589 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d67ba037-b0f2-4445-97ad-fd63463ecfe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:57.571885 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-d67ba037-b0f2-4445-97ad-fd63463ecfe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lazy-loading 'flavor' on Instance uuid db56898a-dff9-4cad-bf8d-f84b53247fc3 {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:06:57.592585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ddf55bb6-feb0-450d-a212-48ed041d77d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:57.601623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ddf55bb6-feb0-450d-a212-48ed041d77d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1319 microversion: 2.1 time: 0.270873 Jun 10 13:06:57.602341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 110/208] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:57 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1319 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:57.932669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6d90d695-fcbc-4dec-8d53-57b632395547 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:57.933329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6d90d695-fcbc-4dec-8d53-57b632395547 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:57.940553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d90d695-fcbc-4dec-8d53-57b632395547 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:57.944412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d90d695-fcbc-4dec-8d53-57b632395547 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:57.944412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d90d695-fcbc-4dec-8d53-57b632395547 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:57.944412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d90d695-fcbc-4dec-8d53-57b632395547 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:58.095174 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6d90d695-fcbc-4dec-8d53-57b632395547 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:58.103050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6d90d695-fcbc-4dec-8d53-57b632395547 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1322 microversion: 2.1 time: 0.174184 Jun 10 13:06:58.103610 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 111/209] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:57 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1322 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:58.193896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-b56b9c62-0ce3-4982-8d62-ab719f423d1d req-60fb0012-3c30-4c01-8b7f-147910d867b9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aeeb9a9c-292c-468a-8d2b-ef677e319ea7", "tag": "4249289a-c974-46ed-9c60-5d0841190f97"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:58.197702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:58.203776 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-b56b9c62-0ce3-4982-8d62-ab719f423d1d req-60fb0012-3c30-4c01-8b7f-147910d867b9 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:58.204214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-b56b9c62-0ce3-4982-8d62-ab719f423d1d req-60fb0012-3c30-4c01-8b7f-147910d867b9 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:58.214983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-b56b9c62-0ce3-4982-8d62-ab719f423d1d req-60fb0012-3c30-4c01-8b7f-147910d867b9 service nova] Creating event network-changed:4249289a-c974-46ed-9c60-5d0841190f97 for instance aeeb9a9c-292c-468a-8d2b-ef677e319ea7 on hv-cinder-maste Jun 10 13:06:58.220264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-b56b9c62-0ce3-4982-8d62-ab719f423d1d req-60fb0012-3c30-4c01-8b7f-147910d867b9 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030047 Jun 10 13:06:58.220868 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 112/210] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:06:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:06:58.268269 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d67ba037-b0f2-4445-97ad-fd63463ecfe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "POST /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/action" status: 202 len: 0 microversion: 2.20 time: 0.781887 Jun 10 13:06:58.268835 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 99/211] 10.222.0.25 () {62 vars in 1318 bytes} [Fri Jun 10 13:06:57 2022] POST /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/action => generated 0 bytes in 784 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Jun 10 13:06:58.277588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0193fe39-ec55-4048-bf79-ab6f81dfb90a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:58.278922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0193fe39-ec55-4048-bf79-ab6f81dfb90a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:58.285404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0193fe39-ec55-4048-bf79-ab6f81dfb90a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:58.285845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0193fe39-ec55-4048-bf79-ab6f81dfb90a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:58.286621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0193fe39-ec55-4048-bf79-ab6f81dfb90a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:58.287046 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0193fe39-ec55-4048-bf79-ab6f81dfb90a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:58.437837 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0193fe39-ec55-4048-bf79-ab6f81dfb90a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:58.446041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0193fe39-ec55-4048-bf79-ab6f81dfb90a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:58.447630 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0193fe39-ec55-4048-bf79-ab6f81dfb90a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:58.449161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0193fe39-ec55-4048-bf79-ab6f81dfb90a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.175812 Jun 10 13:06:58.449714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 113/212] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:06:58 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:06:58.619759 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3c052f70-082a-4a03-8bfd-4902cba03013 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:58.620452 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3c052f70-082a-4a03-8bfd-4902cba03013 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:58.627952 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c052f70-082a-4a03-8bfd-4902cba03013 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:58.628387 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c052f70-082a-4a03-8bfd-4902cba03013 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:58.629279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c052f70-082a-4a03-8bfd-4902cba03013 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:58.629595 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c052f70-082a-4a03-8bfd-4902cba03013 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:58.803785 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3c052f70-082a-4a03-8bfd-4902cba03013 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:58.814858 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3c052f70-082a-4a03-8bfd-4902cba03013 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1319 microversion: 2.1 time: 0.197965 Jun 10 13:06:58.825172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 100/213] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:58 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1319 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:59.121338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2e2123c9-d279-43dd-bc65-289a43ed5f93 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:59.122513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2e2123c9-d279-43dd-bc65-289a43ed5f93 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:59.124419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=8 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:59.129731 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e2123c9-d279-43dd-bc65-289a43ed5f93 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:59.130181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e2123c9-d279-43dd-bc65-289a43ed5f93 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:59.130947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e2123c9-d279-43dd-bc65-289a43ed5f93 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:59.131376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e2123c9-d279-43dd-bc65-289a43ed5f93 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:59.272668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2e2123c9-d279-43dd-bc65-289a43ed5f93 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:59.283538 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2e2123c9-d279-43dd-bc65-289a43ed5f93 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1516 microversion: 2.1 time: 0.166292 Jun 10 13:06:59.284109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 114/214] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:59 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1516 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:06:59.300827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e45c40a7-5c3d-4bde-b8b0-12d53a370f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:06:59.301768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e45c40a7-5c3d-4bde-b8b0-12d53a370f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:59.308616 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e45c40a7-5c3d-4bde-b8b0-12d53a370f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:59.309119 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e45c40a7-5c3d-4bde-b8b0-12d53a370f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:59.309881 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e45c40a7-5c3d-4bde-b8b0-12d53a370f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:59.310481 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e45c40a7-5c3d-4bde-b8b0-12d53a370f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:59.341235 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-e45c40a7-5c3d-4bde-b8b0-12d53a370f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lazy-loading 'flavor' on Instance uuid 6ddb4431-e631-4edd-af56-46cbe0241ca8 {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:06:59.478915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a7856144-6fef-4f83-9dbc-9badcd48921f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:59.480759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a7856144-6fef-4f83-9dbc-9badcd48921f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:59.489635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7856144-6fef-4f83-9dbc-9badcd48921f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:59.490373 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7856144-6fef-4f83-9dbc-9badcd48921f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:59.491477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7856144-6fef-4f83-9dbc-9badcd48921f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:59.492146 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7856144-6fef-4f83-9dbc-9badcd48921f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:59.545370 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e45c40a7-5c3d-4bde-b8b0-12d53a370f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "POST /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8/action" status: 200 len: 32 microversion: 2.1 time: 0.248407 Jun 10 13:06:59.546174 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 101/215] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Jun 10 13:06:59 2022] POST /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8/action => generated 32 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jun 10 13:06:59.556393 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b756d576-4a0d-483b-95b1-835fb116df17 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:59.557407 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b756d576-4a0d-483b-95b1-835fb116df17 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:59.564816 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b756d576-4a0d-483b-95b1-835fb116df17 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:59.565513 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b756d576-4a0d-483b-95b1-835fb116df17 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:59.566287 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b756d576-4a0d-483b-95b1-835fb116df17 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:59.566970 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b756d576-4a0d-483b-95b1-835fb116df17 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:59.649131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a7856144-6fef-4f83-9dbc-9badcd48921f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:59.657771 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a7856144-6fef-4f83-9dbc-9badcd48921f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:59.659272 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a7856144-6fef-4f83-9dbc-9badcd48921f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:59.660662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a7856144-6fef-4f83-9dbc-9badcd48921f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.185923 Jun 10 13:06:59.661017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 115/216] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:06:59 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:06:59.697831 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=2 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:59.795727 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b756d576-4a0d-483b-95b1-835fb116df17 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:06:59.808284 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b756d576-4a0d-483b-95b1-835fb116df17 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.256465 Jun 10 13:06:59.819257 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 102/217] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:59 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:06:59.833276 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c2d6f3a2-6aa4-45fc-b5fa-38729f7dd538 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:06:59.834152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c2d6f3a2-6aa4-45fc-b5fa-38729f7dd538 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:06:59.838658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:59.846901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c2d6f3a2-6aa4-45fc-b5fa-38729f7dd538 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:59.847617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c2d6f3a2-6aa4-45fc-b5fa-38729f7dd538 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:06:59.848596 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c2d6f3a2-6aa4-45fc-b5fa-38729f7dd538 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:06:59.849108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c2d6f3a2-6aa4-45fc-b5fa-38729f7dd538 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:00.021614 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c2d6f3a2-6aa4-45fc-b5fa-38729f7dd538 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:00.025564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c2d6f3a2-6aa4-45fc-b5fa-38729f7dd538 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1319 microversion: 2.1 time: 0.197415 Jun 10 13:07:00.026319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 116/218] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:06:59 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1319 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:00.682742 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3d321c7a-5c57-403f-9c06-bf74b1fc5977 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:00.683438 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3d321c7a-5c57-403f-9c06-bf74b1fc5977 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:00.712172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d321c7a-5c57-403f-9c06-bf74b1fc5977 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:00.712172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d321c7a-5c57-403f-9c06-bf74b1fc5977 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:00.712172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d321c7a-5c57-403f-9c06-bf74b1fc5977 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:00.712172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d321c7a-5c57-403f-9c06-bf74b1fc5977 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:00.830922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9800a47a-9887-4b66-b2b2-21dfb4687a6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:00.831537 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9800a47a-9887-4b66-b2b2-21dfb4687a6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:00.838696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9800a47a-9887-4b66-b2b2-21dfb4687a6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:00.839223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9800a47a-9887-4b66-b2b2-21dfb4687a6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:00.840298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9800a47a-9887-4b66-b2b2-21dfb4687a6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:00.841548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9800a47a-9887-4b66-b2b2-21dfb4687a6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:00.847988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3d321c7a-5c57-403f-9c06-bf74b1fc5977 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:00.856845 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3d321c7a-5c57-403f-9c06-bf74b1fc5977 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:00.862244 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3d321c7a-5c57-403f-9c06-bf74b1fc5977 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:00.863444 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3d321c7a-5c57-403f-9c06-bf74b1fc5977 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.185064 Jun 10 13:07:00.864080 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 103/219] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:00 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:00.990698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9800a47a-9887-4b66-b2b2-21dfb4687a6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:00.993155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:00.998934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9800a47a-9887-4b66-b2b2-21dfb4687a6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.172029 Jun 10 13:07:00.999499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 117/220] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:00 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:01.042506 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-dac4e2bf-9a8f-401b-a8d3-e1290d93086e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:01.043132 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-dac4e2bf-9a8f-401b-a8d3-e1290d93086e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:01.049225 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dac4e2bf-9a8f-401b-a8d3-e1290d93086e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:01.049668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dac4e2bf-9a8f-401b-a8d3-e1290d93086e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:01.050431 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dac4e2bf-9a8f-401b-a8d3-e1290d93086e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:01.050865 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dac4e2bf-9a8f-401b-a8d3-e1290d93086e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:01.261878 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-dac4e2bf-9a8f-401b-a8d3-e1290d93086e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:01.270692 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-dac4e2bf-9a8f-401b-a8d3-e1290d93086e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1319 microversion: 2.1 time: 0.232097 Jun 10 13:07:01.271387 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 104/221] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:01 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1319 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:01.883585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c8d0b38b-7591-48c5-a2a0-860ec005098b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:01.884669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c8d0b38b-7591-48c5-a2a0-860ec005098b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:01.893066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c8d0b38b-7591-48c5-a2a0-860ec005098b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:01.893562 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c8d0b38b-7591-48c5-a2a0-860ec005098b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:01.894351 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c8d0b38b-7591-48c5-a2a0-860ec005098b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:01.894785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c8d0b38b-7591-48c5-a2a0-860ec005098b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:02.018223 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3da8f800-6049-4332-aa3b-c3ba0f2bb94e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:02.019979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3da8f800-6049-4332-aa3b-c3ba0f2bb94e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:02.029076 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3da8f800-6049-4332-aa3b-c3ba0f2bb94e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:02.036892 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3da8f800-6049-4332-aa3b-c3ba0f2bb94e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:02.036892 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3da8f800-6049-4332-aa3b-c3ba0f2bb94e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:02.036892 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3da8f800-6049-4332-aa3b-c3ba0f2bb94e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:02.052799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c8d0b38b-7591-48c5-a2a0-860ec005098b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:02.062089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c8d0b38b-7591-48c5-a2a0-860ec005098b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:02.063750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c8d0b38b-7591-48c5-a2a0-860ec005098b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:02.066852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c8d0b38b-7591-48c5-a2a0-860ec005098b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.188003 Jun 10 13:07:02.073488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 118/222] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:01 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:02.211041 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3da8f800-6049-4332-aa3b-c3ba0f2bb94e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:02.218745 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3da8f800-6049-4332-aa3b-c3ba0f2bb94e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.206763 Jun 10 13:07:02.219307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 105/223] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:02 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:02.291055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d74f1a77-bc8a-4359-82a2-28c44fe9fd35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:02.291665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d74f1a77-bc8a-4359-82a2-28c44fe9fd35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:02.298389 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d74f1a77-bc8a-4359-82a2-28c44fe9fd35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:02.298917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d74f1a77-bc8a-4359-82a2-28c44fe9fd35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:02.299653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d74f1a77-bc8a-4359-82a2-28c44fe9fd35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:02.300088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d74f1a77-bc8a-4359-82a2-28c44fe9fd35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:02.462222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d74f1a77-bc8a-4359-82a2-28c44fe9fd35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:02.470882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d74f1a77-bc8a-4359-82a2-28c44fe9fd35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1319 microversion: 2.1 time: 0.183892 Jun 10 13:07:02.471457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 119/224] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:02 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1319 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:03.084585 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e6a559ce-1726-4875-b6f4-534fcff24592 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:03.085290 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e6a559ce-1726-4875-b6f4-534fcff24592 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:03.092737 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6a559ce-1726-4875-b6f4-534fcff24592 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:03.093219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6a559ce-1726-4875-b6f4-534fcff24592 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:03.093981 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6a559ce-1726-4875-b6f4-534fcff24592 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:03.094409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6a559ce-1726-4875-b6f4-534fcff24592 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:03.225517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e6a559ce-1726-4875-b6f4-534fcff24592 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:03.234338 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e6a559ce-1726-4875-b6f4-534fcff24592 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:03.236154 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e6a559ce-1726-4875-b6f4-534fcff24592 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:03.236325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-325f6fa2-afa4-4532-9925-9c3f87f0fa7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:03.236962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-325f6fa2-afa4-4532-9925-9c3f87f0fa7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:03.237362 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e6a559ce-1726-4875-b6f4-534fcff24592 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.156975 Jun 10 13:07:03.238027 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 106/225] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:03 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:03.243597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-325f6fa2-afa4-4532-9925-9c3f87f0fa7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:03.243974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-325f6fa2-afa4-4532-9925-9c3f87f0fa7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:03.244729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-325f6fa2-afa4-4532-9925-9c3f87f0fa7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:03.245180 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-325f6fa2-afa4-4532-9925-9c3f87f0fa7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:03.383155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-325f6fa2-afa4-4532-9925-9c3f87f0fa7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:03.391283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-325f6fa2-afa4-4532-9925-9c3f87f0fa7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.159617 Jun 10 13:07:03.391847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 120/226] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:03 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:03.490074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e7b80307-f404-4a1e-abdc-9a6a7bd029f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:03.490688 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e7b80307-f404-4a1e-abdc-9a6a7bd029f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:03.496677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e7b80307-f404-4a1e-abdc-9a6a7bd029f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:03.497106 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e7b80307-f404-4a1e-abdc-9a6a7bd029f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:03.497869 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e7b80307-f404-4a1e-abdc-9a6a7bd029f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:03.498300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e7b80307-f404-4a1e-abdc-9a6a7bd029f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:03.629261 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e7b80307-f404-4a1e-abdc-9a6a7bd029f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:03.637246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e7b80307-f404-4a1e-abdc-9a6a7bd029f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1319 microversion: 2.1 time: 0.150886 Jun 10 13:07:03.642272 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 107/227] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:03 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1319 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:04.255298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-da5ce57c-08ab-4de6-a778-7dbf42644013 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:04.255949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-da5ce57c-08ab-4de6-a778-7dbf42644013 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:04.261998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-da5ce57c-08ab-4de6-a778-7dbf42644013 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:04.262441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-da5ce57c-08ab-4de6-a778-7dbf42644013 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:04.263307 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-da5ce57c-08ab-4de6-a778-7dbf42644013 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:04.263741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-da5ce57c-08ab-4de6-a778-7dbf42644013 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:04.409529 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c8e8a5d9-1ee5-46eb-bec9-c9b2cb05ae5e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:04.410023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-da5ce57c-08ab-4de6-a778-7dbf42644013 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:04.410127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c8e8a5d9-1ee5-46eb-bec9-c9b2cb05ae5e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:04.418301 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-da5ce57c-08ab-4de6-a778-7dbf42644013 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:04.420430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-da5ce57c-08ab-4de6-a778-7dbf42644013 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:04.421638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-da5ce57c-08ab-4de6-a778-7dbf42644013 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.170451 Jun 10 13:07:04.422629 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c8e8a5d9-1ee5-46eb-bec9-c9b2cb05ae5e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:04.422890 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 121/228] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:04 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:04.423508 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c8e8a5d9-1ee5-46eb-bec9-c9b2cb05ae5e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:04.424349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c8e8a5d9-1ee5-46eb-bec9-c9b2cb05ae5e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:04.424900 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c8e8a5d9-1ee5-46eb-bec9-c9b2cb05ae5e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:04.558089 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c8e8a5d9-1ee5-46eb-bec9-c9b2cb05ae5e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:04.560286 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=6,nova_cell1:INSERT=3 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:04.566286 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c8e8a5d9-1ee5-46eb-bec9-c9b2cb05ae5e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.160919 Jun 10 13:07:04.566793 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 108/229] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:04 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:04.654946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0da9fc4d-6610-47ed-855e-bf6070528775 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:04.655683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0da9fc4d-6610-47ed-855e-bf6070528775 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:04.662022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0da9fc4d-6610-47ed-855e-bf6070528775 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:04.662468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0da9fc4d-6610-47ed-855e-bf6070528775 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:04.663227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0da9fc4d-6610-47ed-855e-bf6070528775 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:04.663730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0da9fc4d-6610-47ed-855e-bf6070528775 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:04.792235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0da9fc4d-6610-47ed-855e-bf6070528775 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:04.800624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0da9fc4d-6610-47ed-855e-bf6070528775 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1319 microversion: 2.1 time: 0.150132 Jun 10 13:07:04.801265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 122/230] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:04 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1319 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:05.441813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f3a0e010-7560-4e4e-b1dd-d2cc7a97da6e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:05.442423 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f3a0e010-7560-4e4e-b1dd-d2cc7a97da6e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:05.449294 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3a0e010-7560-4e4e-b1dd-d2cc7a97da6e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:05.449747 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3a0e010-7560-4e4e-b1dd-d2cc7a97da6e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:05.450555 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3a0e010-7560-4e4e-b1dd-d2cc7a97da6e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:05.451591 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3a0e010-7560-4e4e-b1dd-d2cc7a97da6e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:05.583451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-437c03a4-49bf-4070-9368-0f121886137e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:05.584065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-437c03a4-49bf-4070-9368-0f121886137e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:05.586305 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f3a0e010-7560-4e4e-b1dd-d2cc7a97da6e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:05.591234 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-437c03a4-49bf-4070-9368-0f121886137e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:05.591605 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-437c03a4-49bf-4070-9368-0f121886137e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:05.592376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-437c03a4-49bf-4070-9368-0f121886137e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:05.592833 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-437c03a4-49bf-4070-9368-0f121886137e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:05.597231 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f3a0e010-7560-4e4e-b1dd-d2cc7a97da6e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:05.597632 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f3a0e010-7560-4e4e-b1dd-d2cc7a97da6e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:05.598801 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f3a0e010-7560-4e4e-b1dd-d2cc7a97da6e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.161399 Jun 10 13:07:05.599493 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 109/231] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:05 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:05.750343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-437c03a4-49bf-4070-9368-0f121886137e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:05.757948 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-437c03a4-49bf-4070-9368-0f121886137e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.178379 Jun 10 13:07:05.758519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 123/232] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:05 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:05.819145 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2c5c991d-8492-4d2e-a308-2639f403c2c8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:05.820079 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2c5c991d-8492-4d2e-a308-2639f403c2c8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:05.826924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c5c991d-8492-4d2e-a308-2639f403c2c8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:05.827602 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c5c991d-8492-4d2e-a308-2639f403c2c8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:05.828541 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c5c991d-8492-4d2e-a308-2639f403c2c8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:05.831083 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c5c991d-8492-4d2e-a308-2639f403c2c8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:05.975543 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2c5c991d-8492-4d2e-a308-2639f403c2c8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:05.985304 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2c5c991d-8492-4d2e-a308-2639f403c2c8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1319 microversion: 2.1 time: 0.170215 Jun 10 13:07:05.985874 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 110/233] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:05 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1319 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:06.296665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d7f310c3-ccc1-4339-b161-739626cf61c7 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:06.303022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d7f310c3-ccc1-4339-b161-739626cf61c7 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:06.303470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d7f310c3-ccc1-4339-b161-739626cf61c7 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:06.373977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d7f310c3-ccc1-4339-b161-739626cf61c7 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Going to try to stop instance {{(pid=118092) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jun 10 13:07:06.563157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d7f310c3-ccc1-4339-b161-739626cf61c7 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "POST /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/action" status: 202 len: 0 microversion: 2.1 time: 0.267542 Jun 10 13:07:06.564137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 124/234] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Jun 10 13:07:06 2022] POST /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/action => generated 0 bytes in 272 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jun 10 13:07:06.572313 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-fdf63c36-1aff-4bb5-8231-2b5a2a7619bf req-ad059dc0-2f6c-4a0e-8b3b-475a6e6de0fe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aeeb9a9c-292c-468a-8d2b-ef677e319ea7", "name": "network-vif-plugged", "status": "completed", "tag": "4249289a-c974-46ed-9c60-5d0841190f97"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:06.572981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5b643bb5-19c4-4103-a2a1-44072d48e505 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:06.573605 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5b643bb5-19c4-4103-a2a1-44072d48e505 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:06.594052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5b643bb5-19c4-4103-a2a1-44072d48e505 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:06.596252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5b643bb5-19c4-4103-a2a1-44072d48e505 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:06.597238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5b643bb5-19c4-4103-a2a1-44072d48e505 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:06.598213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5b643bb5-19c4-4103-a2a1-44072d48e505 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:06.611395 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-fdf63c36-1aff-4bb5-8231-2b5a2a7619bf req-ad059dc0-2f6c-4a0e-8b3b-475a6e6de0fe service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:06.614013 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-fdf63c36-1aff-4bb5-8231-2b5a2a7619bf req-ad059dc0-2f6c-4a0e-8b3b-475a6e6de0fe service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:06.630701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-fdf63c36-1aff-4bb5-8231-2b5a2a7619bf req-ad059dc0-2f6c-4a0e-8b3b-475a6e6de0fe service nova] Creating event network-vif-plugged:4249289a-c974-46ed-9c60-5d0841190f97 for instance aeeb9a9c-292c-468a-8d2b-ef677e319ea7 on hv-cinder-maste Jun 10 13:07:06.644448 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-fdf63c36-1aff-4bb5-8231-2b5a2a7619bf req-ad059dc0-2f6c-4a0e-8b3b-475a6e6de0fe service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.086205 Jun 10 13:07:06.650041 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 111/235] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:07:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:07:06.650172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-84237fa3-4639-4916-8a24-03e758585ff7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:06.650762 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-84237fa3-4639-4916-8a24-03e758585ff7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:06.668253 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:06.670161 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-84237fa3-4639-4916-8a24-03e758585ff7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:06.670788 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-84237fa3-4639-4916-8a24-03e758585ff7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:06.671895 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-84237fa3-4639-4916-8a24-03e758585ff7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:06.672794 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-84237fa3-4639-4916-8a24-03e758585ff7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:06.821240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5b643bb5-19c4-4103-a2a1-44072d48e505 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:06.830846 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5b643bb5-19c4-4103-a2a1-44072d48e505 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1724 microversion: 2.1 time: 0.262686 Jun 10 13:07:06.838402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 125/236] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:06 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1724 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:06.847575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-683fd06c-f385-4479-b6de-cd14651a69ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:06.848186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-683fd06c-f385-4479-b6de-cd14651a69ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:06.857370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-683fd06c-f385-4479-b6de-cd14651a69ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:06.860035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-683fd06c-f385-4479-b6de-cd14651a69ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:06.860035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-683fd06c-f385-4479-b6de-cd14651a69ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:06.860035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-683fd06c-f385-4479-b6de-cd14651a69ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:06.889082 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-84237fa3-4639-4916-8a24-03e758585ff7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:06.900980 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-84237fa3-4639-4916-8a24-03e758585ff7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:06.902672 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-84237fa3-4639-4916-8a24-03e758585ff7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:06.904843 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-84237fa3-4639-4916-8a24-03e758585ff7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.258159 Jun 10 13:07:06.905414 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 112/237] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:06 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:07.003067 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1e58f62e-540b-41a6-a65d-8a49dd1de797 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:07.003850 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1e58f62e-540b-41a6-a65d-8a49dd1de797 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:07.019022 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1e58f62e-540b-41a6-a65d-8a49dd1de797 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:07.019022 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1e58f62e-540b-41a6-a65d-8a49dd1de797 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:07.020643 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1e58f62e-540b-41a6-a65d-8a49dd1de797 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:07.023737 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1e58f62e-540b-41a6-a65d-8a49dd1de797 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:07.064608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-683fd06c-f385-4479-b6de-cd14651a69ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:07.075033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-683fd06c-f385-4479-b6de-cd14651a69ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.235493 Jun 10 13:07:07.076214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 126/238] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:06 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:07.238926 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1e58f62e-540b-41a6-a65d-8a49dd1de797 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:07.248539 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1e58f62e-540b-41a6-a65d-8a49dd1de797 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1319 microversion: 2.1 time: 0.248903 Jun 10 13:07:07.254409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 113/239] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:06 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1319 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:07.848290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5724da54-d6f8-4d22-ab22-a4521d8e729b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:07.848922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5724da54-d6f8-4d22-ab22-a4521d8e729b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:07.855784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5724da54-d6f8-4d22-ab22-a4521d8e729b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:07.856228 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5724da54-d6f8-4d22-ab22-a4521d8e729b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:07.857704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5724da54-d6f8-4d22-ab22-a4521d8e729b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:07.858131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5724da54-d6f8-4d22-ab22-a4521d8e729b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:07.927222 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-862bc053-4758-45e7-addd-a81c405438ae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:07.928407 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-862bc053-4758-45e7-addd-a81c405438ae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:07.937352 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-862bc053-4758-45e7-addd-a81c405438ae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:07.938283 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-862bc053-4758-45e7-addd-a81c405438ae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:07.939257 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-862bc053-4758-45e7-addd-a81c405438ae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:07.939888 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-862bc053-4758-45e7-addd-a81c405438ae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:08.028427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5724da54-d6f8-4d22-ab22-a4521d8e729b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:08.037858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5724da54-d6f8-4d22-ab22-a4521d8e729b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1724 microversion: 2.1 time: 0.193654 Jun 10 13:07:08.052926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 127/240] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:07 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1724 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:08.092881 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7be0356e-f402-409a-8de9-7c4587f4c937 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:08.093755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7be0356e-f402-409a-8de9-7c4587f4c937 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:08.103588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7be0356e-f402-409a-8de9-7c4587f4c937 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:08.104051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7be0356e-f402-409a-8de9-7c4587f4c937 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:08.108369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7be0356e-f402-409a-8de9-7c4587f4c937 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:08.110859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7be0356e-f402-409a-8de9-7c4587f4c937 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:08.119285 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-862bc053-4758-45e7-addd-a81c405438ae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:08.128808 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-862bc053-4758-45e7-addd-a81c405438ae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:08.131243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-862bc053-4758-45e7-addd-a81c405438ae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:08.133202 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-862bc053-4758-45e7-addd-a81c405438ae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.213472 Jun 10 13:07:08.140543 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 114/241] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:07 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:07:08.258053 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7be0356e-f402-409a-8de9-7c4587f4c937 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:08.267021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7be0356e-f402-409a-8de9-7c4587f4c937 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.178343 Jun 10 13:07:08.267632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 128/242] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:08 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:08.270131 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-79a0f93b-38c7-4226-9693-1fe0075f4699 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:08.274683 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-79a0f93b-38c7-4226-9693-1fe0075f4699 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:08.276632 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-79a0f93b-38c7-4226-9693-1fe0075f4699 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:08.277103 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-79a0f93b-38c7-4226-9693-1fe0075f4699 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:08.277850 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-79a0f93b-38c7-4226-9693-1fe0075f4699 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:08.278275 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-79a0f93b-38c7-4226-9693-1fe0075f4699 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:08.409624 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-79a0f93b-38c7-4226-9693-1fe0075f4699 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:08.418329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-79a0f93b-38c7-4226-9693-1fe0075f4699 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1319 microversion: 2.1 time: 0.150655 Jun 10 13:07:08.418329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 115/243] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:08 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1319 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:09.055761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ae23c66a-1a99-4386-b2a7-717b160a76c3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:09.056275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ae23c66a-1a99-4386-b2a7-717b160a76c3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:09.062428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae23c66a-1a99-4386-b2a7-717b160a76c3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:09.062876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae23c66a-1a99-4386-b2a7-717b160a76c3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:09.067301 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae23c66a-1a99-4386-b2a7-717b160a76c3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:09.067301 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae23c66a-1a99-4386-b2a7-717b160a76c3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:09.155058 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-662c83a6-d213-4bbf-aa38-c0b1b3c86c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:09.156396 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-662c83a6-d213-4bbf-aa38-c0b1b3c86c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:09.166315 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-662c83a6-d213-4bbf-aa38-c0b1b3c86c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:09.167827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-662c83a6-d213-4bbf-aa38-c0b1b3c86c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:09.170915 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-662c83a6-d213-4bbf-aa38-c0b1b3c86c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:09.170915 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-662c83a6-d213-4bbf-aa38-c0b1b3c86c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:09.238982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ae23c66a-1a99-4386-b2a7-717b160a76c3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:09.247220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ae23c66a-1a99-4386-b2a7-717b160a76c3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1724 microversion: 2.1 time: 0.195173 Jun 10 13:07:09.247857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 129/244] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:09 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1724 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:09.295190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-754098b0-733d-4a11-b8c9-b134cdd472de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:09.296710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-754098b0-733d-4a11-b8c9-b134cdd472de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:09.303689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-754098b0-733d-4a11-b8c9-b134cdd472de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:09.304179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-754098b0-733d-4a11-b8c9-b134cdd472de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:09.304985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-754098b0-733d-4a11-b8c9-b134cdd472de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:09.305421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-754098b0-733d-4a11-b8c9-b134cdd472de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:09.342674 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-662c83a6-d213-4bbf-aa38-c0b1b3c86c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:09.351094 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-662c83a6-d213-4bbf-aa38-c0b1b3c86c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:09.352593 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-662c83a6-d213-4bbf-aa38-c0b1b3c86c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:09.353943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-662c83a6-d213-4bbf-aa38-c0b1b3c86c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.206526 Jun 10 13:07:09.354499 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 116/245] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:09 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:07:09.432981 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-152135d3-13ec-44b6-bca1-d6881ecc62e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:09.433595 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-152135d3-13ec-44b6-bca1-d6881ecc62e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:09.436034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-754098b0-733d-4a11-b8c9-b134cdd472de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:09.451230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-754098b0-733d-4a11-b8c9-b134cdd472de tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.162443 Jun 10 13:07:09.451813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 130/246] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:09 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:09.467113 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-152135d3-13ec-44b6-bca1-d6881ecc62e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:09.467766 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-152135d3-13ec-44b6-bca1-d6881ecc62e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:09.468544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-152135d3-13ec-44b6-bca1-d6881ecc62e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:09.469260 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-152135d3-13ec-44b6-bca1-d6881ecc62e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:09.597874 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-152135d3-13ec-44b6-bca1-d6881ecc62e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:09.605549 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-152135d3-13ec-44b6-bca1-d6881ecc62e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1509 microversion: 2.1 time: 0.176576 Jun 10 13:07:09.606373 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 117/247] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:09 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1509 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:10.264500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7442b233-7c34-4f7c-988a-7800edc8eacf tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:10.265190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7442b233-7c34-4f7c-988a-7800edc8eacf tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:10.271337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7442b233-7c34-4f7c-988a-7800edc8eacf tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:10.271788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7442b233-7c34-4f7c-988a-7800edc8eacf tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:10.272553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7442b233-7c34-4f7c-988a-7800edc8eacf tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:10.273010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7442b233-7c34-4f7c-988a-7800edc8eacf tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:10.284498 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-860855607", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "networks": [{"uuid": "1965e685-7775-47a2-a72b-2fe332a3bed7"}]}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:10.352813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.network.neutron [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] validate_networks() for [('1965e685-7775-47a2-a72b-2fe332a3bed7', None, None, None, None, None)] {{(pid=118093) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:07:10.429088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7442b233-7c34-4f7c-988a-7800edc8eacf tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:10.437609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7442b233-7c34-4f7c-988a-7800edc8eacf tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1724 microversion: 2.1 time: 0.176872 Jun 10 13:07:10.438191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 131/248] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:10 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1724 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:10.446521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-958f115d-4a12-4fff-bf15-4c8cc45ac66b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:10.447133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-958f115d-4a12-4fff-bf15-4c8cc45ac66b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:10.454989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-958f115d-4a12-4fff-bf15-4c8cc45ac66b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:10.455801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-958f115d-4a12-4fff-bf15-4c8cc45ac66b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:10.456642 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-958f115d-4a12-4fff-bf15-4c8cc45ac66b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:10.457422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-958f115d-4a12-4fff-bf15-4c8cc45ac66b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:10.614008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-958f115d-4a12-4fff-bf15-4c8cc45ac66b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:10.624303 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-958f115d-4a12-4fff-bf15-4c8cc45ac66b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:10.626345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-958f115d-4a12-4fff-bf15-4c8cc45ac66b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:10.629981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-958f115d-4a12-4fff-bf15-4c8cc45ac66b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.190827 Jun 10 13:07:10.632979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 132/249] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:10 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:10.637513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cf980737-204c-4e26-a987-7ccf15228221 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:10.638311 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cf980737-204c-4e26-a987-7ccf15228221 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:10.645542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf980737-204c-4e26-a987-7ccf15228221 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:10.646228 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf980737-204c-4e26-a987-7ccf15228221 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:10.647036 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf980737-204c-4e26-a987-7ccf15228221 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:10.647521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf980737-204c-4e26-a987-7ccf15228221 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:10.668564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:10.769978 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Flavor limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:07:10.770390 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Image limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:07:10.770882 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Flavor pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:07:10.771284 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Image pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:07:10.771703 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:07:10.782500 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:10.783073 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:10.784024 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:10.784604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:10.802424 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Getting quotas for project 0fdf6af645ba4a02a6a7278eb1fdff0d. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:07:10.806815 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Getting quotas for user 24343309fb924981a37899f56f15b8ba and project 0fdf6af645ba4a02a6a7278eb1fdff0d. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:07:10.818069 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cf980737-204c-4e26-a987-7ccf15228221 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:10.818331 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Going to run 1 instances... {{(pid=118093) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:07:10.820559 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118093) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:07:10.827218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cf980737-204c-4e26-a987-7ccf15228221 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.195529 Jun 10 13:07:10.827793 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 133/250] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:10 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:10.973272 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3e4e431a-dc1a-4077-b339-09819fa2cc3b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.692469 Jun 10 13:07:10.974255 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 118/251] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:07:10 2022] POST /compute/v2.1/servers => generated 380 bytes in 694 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jun 10 13:07:10.983952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bf5bca14-23f8-413f-9fcf-800d6a49df59 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:10.984559 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bf5bca14-23f8-413f-9fcf-800d6a49df59 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:11.085024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bf5bca14-23f8-413f-9fcf-800d6a49df59 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:11.092225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bf5bca14-23f8-413f-9fcf-800d6a49df59 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1213 microversion: 2.1 time: 0.111489 Jun 10 13:07:11.093655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 134/252] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:10 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1213 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:11.456325 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f5f9a3bc-b4f5-4d02-bbfb-0627b60bc960 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:11.457168 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f5f9a3bc-b4f5-4d02-bbfb-0627b60bc960 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:11.463662 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f5f9a3bc-b4f5-4d02-bbfb-0627b60bc960 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:11.464108 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f5f9a3bc-b4f5-4d02-bbfb-0627b60bc960 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:11.464968 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f5f9a3bc-b4f5-4d02-bbfb-0627b60bc960 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:11.465402 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f5f9a3bc-b4f5-4d02-bbfb-0627b60bc960 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:11.649640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6f489b7c-7be3-405e-a4ea-83ea22243d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:11.650168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6f489b7c-7be3-405e-a4ea-83ea22243d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:11.657238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6f489b7c-7be3-405e-a4ea-83ea22243d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:11.659817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6f489b7c-7be3-405e-a4ea-83ea22243d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:11.659817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6f489b7c-7be3-405e-a4ea-83ea22243d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:11.659817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6f489b7c-7be3-405e-a4ea-83ea22243d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:11.685839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f5f9a3bc-b4f5-4d02-bbfb-0627b60bc960 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:11.697767 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f5f9a3bc-b4f5-4d02-bbfb-0627b60bc960 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1724 microversion: 2.1 time: 0.246643 Jun 10 13:07:11.699943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 119/253] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:11 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1724 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:11.836918 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6f489b7c-7be3-405e-a4ea-83ea22243d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:11.844408 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-bd1b29da-d1af-4709-adfb-05c595b3cabc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:11.849251 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-bd1b29da-d1af-4709-adfb-05c595b3cabc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:11.849400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6f489b7c-7be3-405e-a4ea-83ea22243d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:11.849946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6f489b7c-7be3-405e-a4ea-83ea22243d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:11.851320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6f489b7c-7be3-405e-a4ea-83ea22243d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.205798 Jun 10 13:07:11.851902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 135/254] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:11 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:07:11.852029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bd1b29da-d1af-4709-adfb-05c595b3cabc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:11.852354 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bd1b29da-d1af-4709-adfb-05c595b3cabc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:11.853375 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bd1b29da-d1af-4709-adfb-05c595b3cabc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:11.853807 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bd1b29da-d1af-4709-adfb-05c595b3cabc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:12.063294 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-bd1b29da-d1af-4709-adfb-05c595b3cabc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:12.072629 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-bd1b29da-d1af-4709-adfb-05c595b3cabc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.232527 Jun 10 13:07:12.083298 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 120/255] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:11 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:12.110432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-08f76bf8-7b96-4324-a93e-0a51bf57f37f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:12.111128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-08f76bf8-7b96-4324-a93e-0a51bf57f37f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:12.163757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-08f76bf8-7b96-4324-a93e-0a51bf57f37f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:12.170976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-08f76bf8-7b96-4324-a93e-0a51bf57f37f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1213 microversion: 2.1 time: 0.065340 Jun 10 13:07:12.171547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 136/256] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:12 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1213 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:12.720489 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-de8a86f4-0b90-42b8-89d7-2518caf0527d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:12.721317 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-de8a86f4-0b90-42b8-89d7-2518caf0527d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:12.730562 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-de8a86f4-0b90-42b8-89d7-2518caf0527d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:12.735117 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-de8a86f4-0b90-42b8-89d7-2518caf0527d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:12.735969 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-de8a86f4-0b90-42b8-89d7-2518caf0527d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:12.736406 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-de8a86f4-0b90-42b8-89d7-2518caf0527d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:12.870880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a3a4b086-43f6-483b-9524-54a72ea5f946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:12.871481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a3a4b086-43f6-483b-9524-54a72ea5f946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:12.878643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3a4b086-43f6-483b-9524-54a72ea5f946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:12.879099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3a4b086-43f6-483b-9524-54a72ea5f946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:12.879873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3a4b086-43f6-483b-9524-54a72ea5f946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:12.880539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3a4b086-43f6-483b-9524-54a72ea5f946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:12.890303 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-de8a86f4-0b90-42b8-89d7-2518caf0527d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:12.902854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-de8a86f4-0b90-42b8-89d7-2518caf0527d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1724 microversion: 2.1 time: 0.190407 Jun 10 13:07:12.905190 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 121/257] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:12 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1724 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:13.032548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a3a4b086-43f6-483b-9524-54a72ea5f946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:13.041446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a3a4b086-43f6-483b-9524-54a72ea5f946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:13.042946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a3a4b086-43f6-483b-9524-54a72ea5f946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:13.044116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a3a4b086-43f6-483b-9524-54a72ea5f946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.177233 Jun 10 13:07:13.044706 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 137/258] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:12 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:07:13.090700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ba4a1f0d-5d94-414c-a0e8-b5e132bec7e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:13.091994 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ba4a1f0d-5d94-414c-a0e8-b5e132bec7e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:13.098062 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ba4a1f0d-5d94-414c-a0e8-b5e132bec7e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:13.100869 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ba4a1f0d-5d94-414c-a0e8-b5e132bec7e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:13.101766 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ba4a1f0d-5d94-414c-a0e8-b5e132bec7e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:13.102294 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ba4a1f0d-5d94-414c-a0e8-b5e132bec7e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:13.188145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-80a9ab58-d774-429b-95d3-65ebef6d3de9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:13.189330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-80a9ab58-d774-429b-95d3-65ebef6d3de9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:13.196305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80a9ab58-d774-429b-95d3-65ebef6d3de9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:13.197090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80a9ab58-d774-429b-95d3-65ebef6d3de9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:13.198050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80a9ab58-d774-429b-95d3-65ebef6d3de9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:13.198662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80a9ab58-d774-429b-95d3-65ebef6d3de9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:13.263824 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ba4a1f0d-5d94-414c-a0e8-b5e132bec7e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:13.271932 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ba4a1f0d-5d94-414c-a0e8-b5e132bec7e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.185045 Jun 10 13:07:13.273465 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 122/259] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:13 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:13.285850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-80a9ab58-d774-429b-95d3-65ebef6d3de9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:13.293984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-80a9ab58-d774-429b-95d3-65ebef6d3de9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1217 microversion: 2.1 time: 0.109921 Jun 10 13:07:13.294638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 138/260] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:13 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1217 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:13.925731 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a2a3a03a-f034-430b-986d-ad55be93860d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:13.926507 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a2a3a03a-f034-430b-986d-ad55be93860d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:13.939099 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a2a3a03a-f034-430b-986d-ad55be93860d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:13.939541 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a2a3a03a-f034-430b-986d-ad55be93860d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:13.940307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a2a3a03a-f034-430b-986d-ad55be93860d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:13.940732 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a2a3a03a-f034-430b-986d-ad55be93860d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:14.065025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b6022d10-57d9-4a30-a354-59b7eeaedad3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:14.065025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b6022d10-57d9-4a30-a354-59b7eeaedad3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:14.071266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b6022d10-57d9-4a30-a354-59b7eeaedad3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:14.071723 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b6022d10-57d9-4a30-a354-59b7eeaedad3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:14.072501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b6022d10-57d9-4a30-a354-59b7eeaedad3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:14.073161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b6022d10-57d9-4a30-a354-59b7eeaedad3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:14.099003 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a2a3a03a-f034-430b-986d-ad55be93860d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:14.107765 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a2a3a03a-f034-430b-986d-ad55be93860d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1724 microversion: 2.1 time: 0.190062 Jun 10 13:07:14.117853 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 123/261] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:13 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1724 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:14.290831 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4e89cd4c-28e3-4cb1-b5c9-c106d8fab6b8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:14.291350 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4e89cd4c-28e3-4cb1-b5c9-c106d8fab6b8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:14.297818 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4e89cd4c-28e3-4cb1-b5c9-c106d8fab6b8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:14.298505 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4e89cd4c-28e3-4cb1-b5c9-c106d8fab6b8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:14.299595 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4e89cd4c-28e3-4cb1-b5c9-c106d8fab6b8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:14.300377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4e89cd4c-28e3-4cb1-b5c9-c106d8fab6b8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:14.366611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b6022d10-57d9-4a30-a354-59b7eeaedad3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:14.375514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b6022d10-57d9-4a30-a354-59b7eeaedad3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:14.377064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b6022d10-57d9-4a30-a354-59b7eeaedad3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:14.378451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b6022d10-57d9-4a30-a354-59b7eeaedad3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.318808 Jun 10 13:07:14.379235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 139/262] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:14 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:14.391514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9104c692-fba4-44a4-9ea6-8086b3dd4608 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:14.392096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9104c692-fba4-44a4-9ea6-8086b3dd4608 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:14.398552 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9104c692-fba4-44a4-9ea6-8086b3dd4608 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:14.399324 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9104c692-fba4-44a4-9ea6-8086b3dd4608 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:14.400115 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9104c692-fba4-44a4-9ea6-8086b3dd4608 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:14.400551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9104c692-fba4-44a4-9ea6-8086b3dd4608 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:14.465387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9104c692-fba4-44a4-9ea6-8086b3dd4608 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:14.472916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9104c692-fba4-44a4-9ea6-8086b3dd4608 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1217 microversion: 2.1 time: 0.090102 Jun 10 13:07:14.473493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 140/263] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:14 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1217 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:14.530366 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4e89cd4c-28e3-4cb1-b5c9-c106d8fab6b8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:14.556107 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4e89cd4c-28e3-4cb1-b5c9-c106d8fab6b8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.256517 Jun 10 13:07:14.556107 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 124/264] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:14 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:15.124834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ef16ece0-473b-4080-be36-bde9a00d0570 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:15.125601 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ef16ece0-473b-4080-be36-bde9a00d0570 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:15.131854 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ef16ece0-473b-4080-be36-bde9a00d0570 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:15.132306 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ef16ece0-473b-4080-be36-bde9a00d0570 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:15.138919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ef16ece0-473b-4080-be36-bde9a00d0570 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:15.138919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ef16ece0-473b-4080-be36-bde9a00d0570 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:15.402167 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-bb661501-5ee4-499e-a704-4a454a465ec7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:15.404485 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-bb661501-5ee4-499e-a704-4a454a465ec7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:15.411431 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bb661501-5ee4-499e-a704-4a454a465ec7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:15.411876 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bb661501-5ee4-499e-a704-4a454a465ec7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:15.414732 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bb661501-5ee4-499e-a704-4a454a465ec7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:15.414732 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bb661501-5ee4-499e-a704-4a454a465ec7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:15.587109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ef16ece0-473b-4080-be36-bde9a00d0570 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:15.591098 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-bb661501-5ee4-499e-a704-4a454a465ec7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:15.608998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ef16ece0-473b-4080-be36-bde9a00d0570 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1724 microversion: 2.1 time: 0.487739 Jun 10 13:07:15.612597 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-bb661501-5ee4-499e-a704-4a454a465ec7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:15.614608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 141/265] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:15 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1724 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:15.617204 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-bb661501-5ee4-499e-a704-4a454a465ec7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:15.618687 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-bb661501-5ee4-499e-a704-4a454a465ec7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.220796 Jun 10 13:07:15.634742 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 125/266] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:15 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:07:15.637465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6aecb907-cfcb-46ff-b051-9c034f1a16df tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:15.640592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6aecb907-cfcb-46ff-b051-9c034f1a16df tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:15.647277 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-14b7dde9-4e75-4b54-95df-64350aab72d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:15.648191 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-14b7dde9-4e75-4b54-95df-64350aab72d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:15.658361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6aecb907-cfcb-46ff-b051-9c034f1a16df tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:15.663206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6aecb907-cfcb-46ff-b051-9c034f1a16df tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:15.663576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6aecb907-cfcb-46ff-b051-9c034f1a16df tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:15.664018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6aecb907-cfcb-46ff-b051-9c034f1a16df tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:15.670257 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14b7dde9-4e75-4b54-95df-64350aab72d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:15.676561 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14b7dde9-4e75-4b54-95df-64350aab72d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:15.677473 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14b7dde9-4e75-4b54-95df-64350aab72d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:15.677929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14b7dde9-4e75-4b54-95df-64350aab72d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:15.761834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6aecb907-cfcb-46ff-b051-9c034f1a16df tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:15.769611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6aecb907-cfcb-46ff-b051-9c034f1a16df tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1209 microversion: 2.1 time: 0.153655 Jun 10 13:07:15.770187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 142/267] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:15 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1209 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:15.879571 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-14b7dde9-4e75-4b54-95df-64350aab72d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:15.879571 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:15.884596 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-14b7dde9-4e75-4b54-95df-64350aab72d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.248025 Jun 10 13:07:15.885262 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 126/268] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:15 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:16.628778 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0547bf3e-c058-4d4a-a339-42ffc5518667 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:16.630362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0547bf3e-c058-4d4a-a339-42ffc5518667 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:16.640505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0547bf3e-c058-4d4a-a339-42ffc5518667 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:16.641423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0547bf3e-c058-4d4a-a339-42ffc5518667 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:16.641962 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-748ee1aa-25d1-47b1-94a6-308d4259adc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:16.642677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-748ee1aa-25d1-47b1-94a6-308d4259adc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:16.643123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0547bf3e-c058-4d4a-a339-42ffc5518667 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:16.643702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0547bf3e-c058-4d4a-a339-42ffc5518667 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:16.659509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-748ee1aa-25d1-47b1-94a6-308d4259adc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:16.660244 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-748ee1aa-25d1-47b1-94a6-308d4259adc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:16.662321 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-748ee1aa-25d1-47b1-94a6-308d4259adc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:16.664066 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-748ee1aa-25d1-47b1-94a6-308d4259adc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:16.977815 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0547bf3e-c058-4d4a-a339-42ffc5518667 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:16.990302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0547bf3e-c058-4d4a-a339-42ffc5518667 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1724 microversion: 2.1 time: 0.367310 Jun 10 13:07:17.007929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 143/269] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:16 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1724 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:17.014002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f320608c-b061-4a98-a877-ba2a9c3ec8f6 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:17.014937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f320608c-b061-4a98-a877-ba2a9c3ec8f6 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:17.026067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f320608c-b061-4a98-a877-ba2a9c3ec8f6 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:17.026814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f320608c-b061-4a98-a877-ba2a9c3ec8f6 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:17.028057 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f320608c-b061-4a98-a877-ba2a9c3ec8f6 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:17.028729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f320608c-b061-4a98-a877-ba2a9c3ec8f6 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:17.159591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f320608c-b061-4a98-a877-ba2a9c3ec8f6 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:17.176436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f320608c-b061-4a98-a877-ba2a9c3ec8f6 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1283 microversion: 2.1 time: 0.166573 Jun 10 13:07:17.176649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 144/270] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:17 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1283 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:17.197944 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9f2082bd-bfb6-4879-aff9-3277a085f6a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:17.202220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9f2082bd-bfb6-4879-aff9-3277a085f6a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:17.208832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9f2082bd-bfb6-4879-aff9-3277a085f6a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:17.209473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9f2082bd-bfb6-4879-aff9-3277a085f6a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:17.213143 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9f2082bd-bfb6-4879-aff9-3277a085f6a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:17.213143 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9f2082bd-bfb6-4879-aff9-3277a085f6a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:17.404553 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-748ee1aa-25d1-47b1-94a6-308d4259adc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:17.416014 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-748ee1aa-25d1-47b1-94a6-308d4259adc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:17.418633 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-748ee1aa-25d1-47b1-94a6-308d4259adc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:17.420713 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-748ee1aa-25d1-47b1-94a6-308d4259adc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.784361 Jun 10 13:07:17.428844 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 127/271] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:16 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 791 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:17.459180 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9f2082bd-bfb6-4879-aff9-3277a085f6a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:17.467560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9f2082bd-bfb6-4879-aff9-3277a085f6a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.277216 Jun 10 13:07:17.469894 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 145/272] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:17 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:18.009647 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-59b84646-f593-4e9c-a7fe-4ba97cd42e92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:18.010891 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-59b84646-f593-4e9c-a7fe-4ba97cd42e92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:18.021377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59b84646-f593-4e9c-a7fe-4ba97cd42e92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:18.023284 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59b84646-f593-4e9c-a7fe-4ba97cd42e92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:18.024361 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59b84646-f593-4e9c-a7fe-4ba97cd42e92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:18.027467 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59b84646-f593-4e9c-a7fe-4ba97cd42e92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:18.194846 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dff3d2a3-55e9-4765-b282-14d8e8b70394 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:18.196463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dff3d2a3-55e9-4765-b282-14d8e8b70394 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:18.206309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dff3d2a3-55e9-4765-b282-14d8e8b70394 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:18.207059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dff3d2a3-55e9-4765-b282-14d8e8b70394 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:18.208043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dff3d2a3-55e9-4765-b282-14d8e8b70394 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:18.209422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dff3d2a3-55e9-4765-b282-14d8e8b70394 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:18.290618 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-59b84646-f593-4e9c-a7fe-4ba97cd42e92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:18.300438 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-59b84646-f593-4e9c-a7fe-4ba97cd42e92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1724 microversion: 2.1 time: 0.295470 Jun 10 13:07:18.301476 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 128/273] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:18 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1724 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:18.366403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dff3d2a3-55e9-4765-b282-14d8e8b70394 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:18.376513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dff3d2a3-55e9-4765-b282-14d8e8b70394 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1271 microversion: 2.1 time: 0.186084 Jun 10 13:07:18.377670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 146/274] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:18 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1271 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:18.439168 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-efeb6aff-f3e7-48c4-b61f-411ef1cfbf75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:18.439814 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-efeb6aff-f3e7-48c4-b61f-411ef1cfbf75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:18.451614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-efeb6aff-f3e7-48c4-b61f-411ef1cfbf75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:18.452946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-efeb6aff-f3e7-48c4-b61f-411ef1cfbf75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:18.453986 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-efeb6aff-f3e7-48c4-b61f-411ef1cfbf75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:18.454760 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-efeb6aff-f3e7-48c4-b61f-411ef1cfbf75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:18.486531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f018b818-62ce-4ce1-8513-5985ffc8def9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:18.487570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f018b818-62ce-4ce1-8513-5985ffc8def9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:18.501547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f018b818-62ce-4ce1-8513-5985ffc8def9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:18.502374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f018b818-62ce-4ce1-8513-5985ffc8def9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:18.503764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f018b818-62ce-4ce1-8513-5985ffc8def9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:18.504562 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f018b818-62ce-4ce1-8513-5985ffc8def9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:18.752289 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-efeb6aff-f3e7-48c4-b61f-411ef1cfbf75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:18.764776 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-efeb6aff-f3e7-48c4-b61f-411ef1cfbf75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:18.769941 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-efeb6aff-f3e7-48c4-b61f-411ef1cfbf75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:18.771805 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-efeb6aff-f3e7-48c4-b61f-411ef1cfbf75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.336395 Jun 10 13:07:18.784583 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 129/275] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:18 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:18.823331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f018b818-62ce-4ce1-8513-5985ffc8def9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:18.832911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f018b818-62ce-4ce1-8513-5985ffc8def9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.350388 Jun 10 13:07:18.834242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 147/276] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:18 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:19.323645 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f3f4442e-c3ae-4c18-be6d-bb27d4617861 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:19.325037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f3f4442e-c3ae-4c18-be6d-bb27d4617861 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:19.336751 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3f4442e-c3ae-4c18-be6d-bb27d4617861 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:19.337634 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3f4442e-c3ae-4c18-be6d-bb27d4617861 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:19.339285 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3f4442e-c3ae-4c18-be6d-bb27d4617861 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:19.339887 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3f4442e-c3ae-4c18-be6d-bb27d4617861 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:19.395254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3934fa7c-1db9-4cb3-956d-01778aaae346 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:19.396590 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3934fa7c-1db9-4cb3-956d-01778aaae346 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:19.405484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3934fa7c-1db9-4cb3-956d-01778aaae346 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:19.416469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3934fa7c-1db9-4cb3-956d-01778aaae346 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:19.417665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3934fa7c-1db9-4cb3-956d-01778aaae346 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:19.418305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3934fa7c-1db9-4cb3-956d-01778aaae346 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:19.711600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f3f4442e-c3ae-4c18-be6d-bb27d4617861 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:19.725146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f3f4442e-c3ae-4c18-be6d-bb27d4617861 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1724 microversion: 2.1 time: 0.405536 Jun 10 13:07:19.725874 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 130/277] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:19 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1724 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:19.738643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3934fa7c-1db9-4cb3-956d-01778aaae346 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:19.755776 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3934fa7c-1db9-4cb3-956d-01778aaae346 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1313 microversion: 2.1 time: 0.362185 Jun 10 13:07:19.757009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 148/278] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:19 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1313 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:19.793747 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ffa2bc60-346c-469d-8123-3e7bead7e5b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:19.795731 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ffa2bc60-346c-469d-8123-3e7bead7e5b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:19.815530 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ffa2bc60-346c-469d-8123-3e7bead7e5b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:19.819356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ffa2bc60-346c-469d-8123-3e7bead7e5b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:19.822007 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ffa2bc60-346c-469d-8123-3e7bead7e5b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:19.822683 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ffa2bc60-346c-469d-8123-3e7bead7e5b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:19.852212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8dbb9e33-38b7-4417-9681-259f2d258c7a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:19.853114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8dbb9e33-38b7-4417-9681-259f2d258c7a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:19.866670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8dbb9e33-38b7-4417-9681-259f2d258c7a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:19.868311 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8dbb9e33-38b7-4417-9681-259f2d258c7a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:19.869835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8dbb9e33-38b7-4417-9681-259f2d258c7a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:19.873522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8dbb9e33-38b7-4417-9681-259f2d258c7a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:20.172730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8dbb9e33-38b7-4417-9681-259f2d258c7a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:20.180505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8dbb9e33-38b7-4417-9681-259f2d258c7a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.332719 Jun 10 13:07:20.185525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 149/279] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:19 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:20.230708 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ffa2bc60-346c-469d-8123-3e7bead7e5b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:20.238737 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ffa2bc60-346c-469d-8123-3e7bead7e5b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:20.240481 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ffa2bc60-346c-469d-8123-3e7bead7e5b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:20.241977 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ffa2bc60-346c-469d-8123-3e7bead7e5b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1722 microversion: 2.20 time: 0.453661 Jun 10 13:07:20.243037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 131/280] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:19 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1722 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:20.507637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-1746c9dc-301c-4a88-ae4e-c755b848b143 req-3a46a0cb-daec-4760-ac58-76b35a131ce2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "be7b7cb1-c27c-410a-8434-271566df4aba", "tag": "09e26ef0-4ce5-4d09-bbbf-e96cf7784092"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:20.535087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-1746c9dc-301c-4a88-ae4e-c755b848b143 req-3a46a0cb-daec-4760-ac58-76b35a131ce2 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:20.535680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-1746c9dc-301c-4a88-ae4e-c755b848b143 req-3a46a0cb-daec-4760-ac58-76b35a131ce2 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:20.564921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-1746c9dc-301c-4a88-ae4e-c755b848b143 req-3a46a0cb-daec-4760-ac58-76b35a131ce2 service nova] Creating event network-changed:09e26ef0-4ce5-4d09-bbbf-e96cf7784092 for instance be7b7cb1-c27c-410a-8434-271566df4aba on hv-cinder-maste Jun 10 13:07:20.571866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-1746c9dc-301c-4a88-ae4e-c755b848b143 req-3a46a0cb-daec-4760-ac58-76b35a131ce2 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069402 Jun 10 13:07:20.572819 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 150/281] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:07:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:07:20.751708 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b13bdbb1-85b0-4b44-8569-698425e09745 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:20.753598 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b13bdbb1-85b0-4b44-8569-698425e09745 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:20.775065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5fe9589d-8ce2-4664-8b16-265c101e2742 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:20.775782 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5fe9589d-8ce2-4664-8b16-265c101e2742 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:20.781486 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b13bdbb1-85b0-4b44-8569-698425e09745 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:20.788724 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b13bdbb1-85b0-4b44-8569-698425e09745 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:20.790563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5fe9589d-8ce2-4664-8b16-265c101e2742 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:20.794129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5fe9589d-8ce2-4664-8b16-265c101e2742 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:20.794129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5fe9589d-8ce2-4664-8b16-265c101e2742 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:20.794976 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b13bdbb1-85b0-4b44-8569-698425e09745 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:20.794976 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b13bdbb1-85b0-4b44-8569-698425e09745 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:20.794976 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=2 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:20.800831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5fe9589d-8ce2-4664-8b16-265c101e2742 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:20.881015 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=6 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:21.129484 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b13bdbb1-85b0-4b44-8569-698425e09745 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:21.151022 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b13bdbb1-85b0-4b44-8569-698425e09745 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1724 microversion: 2.1 time: 0.411150 Jun 10 13:07:21.151674 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 132/282] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:20 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1724 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:21.162557 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5fe9589d-8ce2-4664-8b16-265c101e2742 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:21.173634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5fe9589d-8ce2-4664-8b16-265c101e2742 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1313 microversion: 2.1 time: 0.403885 Jun 10 13:07:21.174220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 151/283] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:20 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1313 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:21.206826 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9548e33e-ddd7-4476-8dfe-16c0e472c3f4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:21.206826 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9548e33e-ddd7-4476-8dfe-16c0e472c3f4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:21.212214 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9548e33e-ddd7-4476-8dfe-16c0e472c3f4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:21.213109 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9548e33e-ddd7-4476-8dfe-16c0e472c3f4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:21.214326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9548e33e-ddd7-4476-8dfe-16c0e472c3f4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:21.215044 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9548e33e-ddd7-4476-8dfe-16c0e472c3f4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:21.260901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-64d5f667-2bdf-4178-bd45-3a26e2977782 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:21.261728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-64d5f667-2bdf-4178-bd45-3a26e2977782 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:21.268897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-64d5f667-2bdf-4178-bd45-3a26e2977782 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:21.269347 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-64d5f667-2bdf-4178-bd45-3a26e2977782 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:21.270127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-64d5f667-2bdf-4178-bd45-3a26e2977782 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:21.270545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-64d5f667-2bdf-4178-bd45-3a26e2977782 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:21.446427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-64d5f667-2bdf-4178-bd45-3a26e2977782 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:21.449421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:21.454068 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9548e33e-ddd7-4476-8dfe-16c0e472c3f4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:21.461787 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9548e33e-ddd7-4476-8dfe-16c0e472c3f4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.266316 Jun 10 13:07:21.462619 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 133/284] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:21 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:21.463733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-64d5f667-2bdf-4178-bd45-3a26e2977782 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:21.465257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-64d5f667-2bdf-4178-bd45-3a26e2977782 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:21.467078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-64d5f667-2bdf-4178-bd45-3a26e2977782 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1743 microversion: 2.20 time: 0.209686 Jun 10 13:07:21.467811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 152/285] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:21 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1743 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:07:22.170075 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-28f28b44-a474-4d79-919d-f64b18633695 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:22.170688 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-28f28b44-a474-4d79-919d-f64b18633695 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:22.177407 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-28f28b44-a474-4d79-919d-f64b18633695 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:22.177857 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-28f28b44-a474-4d79-919d-f64b18633695 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:22.178808 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-28f28b44-a474-4d79-919d-f64b18633695 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:22.179252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-28f28b44-a474-4d79-919d-f64b18633695 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:22.191145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-848da8a0-3871-4e55-836a-2164365eb58f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:22.192331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-848da8a0-3871-4e55-836a-2164365eb58f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:22.199719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=4 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:22.199719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-848da8a0-3871-4e55-836a-2164365eb58f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:22.200463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-848da8a0-3871-4e55-836a-2164365eb58f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:22.201965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-848da8a0-3871-4e55-836a-2164365eb58f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:22.202561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-848da8a0-3871-4e55-836a-2164365eb58f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:22.393589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-848da8a0-3871-4e55-836a-2164365eb58f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:22.402990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-848da8a0-3871-4e55-836a-2164365eb58f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1313 microversion: 2.1 time: 0.215858 Jun 10 13:07:22.413970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 153/286] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:22 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1313 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:22.480237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e46e78b0-6226-4922-ad5f-44d46e4d55fa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:22.481923 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e46e78b0-6226-4922-ad5f-44d46e4d55fa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:22.487926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e46e78b0-6226-4922-ad5f-44d46e4d55fa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:22.488492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e46e78b0-6226-4922-ad5f-44d46e4d55fa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:22.489473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e46e78b0-6226-4922-ad5f-44d46e4d55fa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:22.490147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e46e78b0-6226-4922-ad5f-44d46e4d55fa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:22.503845 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-28f28b44-a474-4d79-919d-f64b18633695 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:22.519047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-28f28b44-a474-4d79-919d-f64b18633695 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1724 microversion: 2.1 time: 0.352791 Jun 10 13:07:22.529436 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 134/287] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:22 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1724 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:22.534382 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ae5ed9e7-60fb-4433-8050-f52905077987 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:22.535296 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ae5ed9e7-60fb-4433-8050-f52905077987 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:22.541913 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ae5ed9e7-60fb-4433-8050-f52905077987 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:22.542617 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ae5ed9e7-60fb-4433-8050-f52905077987 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:22.543612 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ae5ed9e7-60fb-4433-8050-f52905077987 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:22.544209 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ae5ed9e7-60fb-4433-8050-f52905077987 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:22.675702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e46e78b0-6226-4922-ad5f-44d46e4d55fa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:22.687017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e46e78b0-6226-4922-ad5f-44d46e4d55fa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.210738 Jun 10 13:07:22.689836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 154/288] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:22 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:22.799730 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ae5ed9e7-60fb-4433-8050-f52905077987 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:22.812533 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ae5ed9e7-60fb-4433-8050-f52905077987 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:22.818436 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ae5ed9e7-60fb-4433-8050-f52905077987 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:22.818436 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ae5ed9e7-60fb-4433-8050-f52905077987 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1743 microversion: 2.20 time: 0.285038 Jun 10 13:07:22.818436 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 135/289] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:22 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1743 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:23.422933 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cecf169b-08ae-462d-9dad-f7a759f82fd1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:23.425298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cecf169b-08ae-462d-9dad-f7a759f82fd1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:23.430196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cecf169b-08ae-462d-9dad-f7a759f82fd1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:23.430653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cecf169b-08ae-462d-9dad-f7a759f82fd1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:23.431626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cecf169b-08ae-462d-9dad-f7a759f82fd1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:23.432072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cecf169b-08ae-462d-9dad-f7a759f82fd1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:23.537689 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d90f0c2a-7723-4189-86df-7ca276535087 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:23.539110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d90f0c2a-7723-4189-86df-7ca276535087 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:23.546766 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d90f0c2a-7723-4189-86df-7ca276535087 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:23.547248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d90f0c2a-7723-4189-86df-7ca276535087 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:23.548827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d90f0c2a-7723-4189-86df-7ca276535087 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:23.549285 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d90f0c2a-7723-4189-86df-7ca276535087 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:23.649500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cecf169b-08ae-462d-9dad-f7a759f82fd1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:23.661459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cecf169b-08ae-462d-9dad-f7a759f82fd1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1313 microversion: 2.1 time: 0.243602 Jun 10 13:07:23.663733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 155/290] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:23 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1313 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:23.706657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-52e2c1c7-38d9-4d6e-9829-733b1f9de52a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:23.707393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-52e2c1c7-38d9-4d6e-9829-733b1f9de52a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:23.713746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52e2c1c7-38d9-4d6e-9829-733b1f9de52a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:23.714280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52e2c1c7-38d9-4d6e-9829-733b1f9de52a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:23.715298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52e2c1c7-38d9-4d6e-9829-733b1f9de52a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:23.715740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52e2c1c7-38d9-4d6e-9829-733b1f9de52a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:23.788002 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d90f0c2a-7723-4189-86df-7ca276535087 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:23.808676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d90f0c2a-7723-4189-86df-7ca276535087 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1701 microversion: 2.1 time: 0.275426 Jun 10 13:07:23.822702 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 136/291] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:23 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1701 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:23.835223 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-27a001fb-cb6d-4c97-82bc-d775f06b66f1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:23.844298 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-27a001fb-cb6d-4c97-82bc-d775f06b66f1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:23.845132 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-27a001fb-cb6d-4c97-82bc-d775f06b66f1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:23.915693 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-52e2c1c7-38d9-4d6e-9829-733b1f9de52a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:23.948344 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-52e2c1c7-38d9-4d6e-9829-733b1f9de52a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.245941 Jun 10 13:07:23.955373 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 156/292] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:23 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:23.965805 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-071f548b-0cc5-47aa-a83e-8277d2d166f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:23.973847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-071f548b-0cc5-47aa-a83e-8277d2d166f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:23.974317 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-27a001fb-cb6d-4c97-82bc-d775f06b66f1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Going to try to start instance {{(pid=118093) start /opt/stack/nova/nova/compute/api.py:2730}} Jun 10 13:07:23.979790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-071f548b-0cc5-47aa-a83e-8277d2d166f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:23.982433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-071f548b-0cc5-47aa-a83e-8277d2d166f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:23.983327 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-071f548b-0cc5-47aa-a83e-8277d2d166f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:23.984732 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-071f548b-0cc5-47aa-a83e-8277d2d166f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:24.276575 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-27a001fb-cb6d-4c97-82bc-d775f06b66f1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "POST /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/action" status: 202 len: 0 microversion: 2.1 time: 0.447502 Jun 10 13:07:24.276575 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 137/293] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Jun 10 13:07:23 2022] POST /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/action => generated 0 bytes in 449 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jun 10 13:07:24.284510 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-dda181af-3a05-4c4b-9351-dac188b05220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:24.285499 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-dda181af-3a05-4c4b-9351-dac188b05220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:24.300227 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dda181af-3a05-4c4b-9351-dac188b05220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:24.301001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dda181af-3a05-4c4b-9351-dac188b05220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:24.303303 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dda181af-3a05-4c4b-9351-dac188b05220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:24.304201 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dda181af-3a05-4c4b-9351-dac188b05220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:24.454847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-071f548b-0cc5-47aa-a83e-8277d2d166f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:24.463161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-071f548b-0cc5-47aa-a83e-8277d2d166f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:24.491180 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-071f548b-0cc5-47aa-a83e-8277d2d166f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:24.492742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-071f548b-0cc5-47aa-a83e-8277d2d166f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1743 microversion: 2.20 time: 0.535951 Jun 10 13:07:24.503903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 157/294] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:23 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1743 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:24.622458 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-dda181af-3a05-4c4b-9351-dac188b05220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:24.636996 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-dda181af-3a05-4c4b-9351-dac188b05220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1710 microversion: 2.1 time: 0.357233 Jun 10 13:07:24.641368 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 138/295] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:24 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1710 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:24.686497 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8dafd57a-d8bb-4e88-87ca-3045ddf3ebef tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:24.686497 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8dafd57a-d8bb-4e88-87ca-3045ddf3ebef tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:24.691375 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8dafd57a-d8bb-4e88-87ca-3045ddf3ebef tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:24.693262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8dafd57a-d8bb-4e88-87ca-3045ddf3ebef tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:24.694104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8dafd57a-d8bb-4e88-87ca-3045ddf3ebef tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:24.694700 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8dafd57a-d8bb-4e88-87ca-3045ddf3ebef tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:24.965875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-69fccc35-0cce-4a1a-803a-b43a5e29b2a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:24.968567 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-69fccc35-0cce-4a1a-803a-b43a5e29b2a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:24.995539 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69fccc35-0cce-4a1a-803a-b43a5e29b2a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:25.001750 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69fccc35-0cce-4a1a-803a-b43a5e29b2a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:25.001750 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69fccc35-0cce-4a1a-803a-b43a5e29b2a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:25.001750 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69fccc35-0cce-4a1a-803a-b43a5e29b2a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:25.028463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8dafd57a-d8bb-4e88-87ca-3045ddf3ebef tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:25.041532 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8dafd57a-d8bb-4e88-87ca-3045ddf3ebef tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1313 microversion: 2.1 time: 0.362884 Jun 10 13:07:25.042914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 158/296] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:24 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1313 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:25.420107 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-69fccc35-0cce-4a1a-803a-b43a5e29b2a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:25.429716 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-69fccc35-0cce-4a1a-803a-b43a5e29b2a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.467496 Jun 10 13:07:25.431681 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 139/297] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:24 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:25.522643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ae72f6ab-8b91-4b2a-b650-1455da812c17 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:25.523310 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ae72f6ab-8b91-4b2a-b650-1455da812c17 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:25.530612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae72f6ab-8b91-4b2a-b650-1455da812c17 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:25.531904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae72f6ab-8b91-4b2a-b650-1455da812c17 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:25.533090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae72f6ab-8b91-4b2a-b650-1455da812c17 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:25.533914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae72f6ab-8b91-4b2a-b650-1455da812c17 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:25.618434 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-a60bed5d-9039-42b1-bed7-bdd5f3601c72 req-8936d3b8-6905-4098-8b8b-144c2952607c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "db56898a-dff9-4cad-bf8d-f84b53247fc3", "name": "network-vif-plugged", "status": "completed", "tag": "31e1a9d0-3f1e-4221-a431-639f0e4a6ff4"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:25.638783 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-a60bed5d-9039-42b1-bed7-bdd5f3601c72 req-8936d3b8-6905-4098-8b8b-144c2952607c service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:25.640020 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-a60bed5d-9039-42b1-bed7-bdd5f3601c72 req-8936d3b8-6905-4098-8b8b-144c2952607c service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:25.657438 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-a60bed5d-9039-42b1-bed7-bdd5f3601c72 req-8936d3b8-6905-4098-8b8b-144c2952607c service nova] Creating event network-vif-plugged:31e1a9d0-3f1e-4221-a431-639f0e4a6ff4 for instance db56898a-dff9-4cad-bf8d-f84b53247fc3 on hv-cinder-maste Jun 10 13:07:25.667619 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-a60bed5d-9039-42b1-bed7-bdd5f3601c72 req-8936d3b8-6905-4098-8b8b-144c2952607c service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053908 Jun 10 13:07:25.670220 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 140/298] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:07:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:07:25.677746 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c0553c83-4be0-4d70-924a-abab678baf88 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:25.678700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c0553c83-4be0-4d70-924a-abab678baf88 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:25.698304 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c0553c83-4be0-4d70-924a-abab678baf88 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:25.699307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c0553c83-4be0-4d70-924a-abab678baf88 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:25.700344 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c0553c83-4be0-4d70-924a-abab678baf88 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:25.702284 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c0553c83-4be0-4d70-924a-abab678baf88 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:25.943669 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c0553c83-4be0-4d70-924a-abab678baf88 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:25.955269 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c0553c83-4be0-4d70-924a-abab678baf88 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1710 microversion: 2.1 time: 0.283537 Jun 10 13:07:25.957143 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 141/299] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:25 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1710 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:26.067212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-af123b2c-aba9-4950-ad30-b775cfc36876 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:26.067930 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-af123b2c-aba9-4950-ad30-b775cfc36876 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:26.085677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-af123b2c-aba9-4950-ad30-b775cfc36876 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:26.090461 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-af123b2c-aba9-4950-ad30-b775cfc36876 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:26.094699 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-af123b2c-aba9-4950-ad30-b775cfc36876 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:26.095524 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-af123b2c-aba9-4950-ad30-b775cfc36876 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:26.205427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ae72f6ab-8b91-4b2a-b650-1455da812c17 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:26.220034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ae72f6ab-8b91-4b2a-b650-1455da812c17 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:26.224157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ae72f6ab-8b91-4b2a-b650-1455da812c17 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:26.227129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ae72f6ab-8b91-4b2a-b650-1455da812c17 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1738 microversion: 2.20 time: 0.708889 Jun 10 13:07:26.233574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 159/300] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:25 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1738 bytes in 715 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:26.447746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-daccdc9a-38f5-4089-a90d-0e7e6127f1a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:26.454224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-daccdc9a-38f5-4089-a90d-0e7e6127f1a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:26.469503 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-daccdc9a-38f5-4089-a90d-0e7e6127f1a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:26.477117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-daccdc9a-38f5-4089-a90d-0e7e6127f1a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:26.478080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-daccdc9a-38f5-4089-a90d-0e7e6127f1a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:26.478525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-daccdc9a-38f5-4089-a90d-0e7e6127f1a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:26.527457 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-af123b2c-aba9-4950-ad30-b775cfc36876 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:26.551628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-af123b2c-aba9-4950-ad30-b775cfc36876 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1313 microversion: 2.1 time: 0.495836 Jun 10 13:07:26.570400 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 142/301] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:26 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1313 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:26.886867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-daccdc9a-38f5-4089-a90d-0e7e6127f1a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:26.896041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-daccdc9a-38f5-4089-a90d-0e7e6127f1a6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.452315 Jun 10 13:07:26.897254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 160/302] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:26 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:26.975124 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b2293cf7-5721-4078-92d3-3b2f7c6b7041 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:26.976308 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b2293cf7-5721-4078-92d3-3b2f7c6b7041 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:26.990549 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b2293cf7-5721-4078-92d3-3b2f7c6b7041 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:26.991302 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b2293cf7-5721-4078-92d3-3b2f7c6b7041 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:26.992467 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b2293cf7-5721-4078-92d3-3b2f7c6b7041 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:26.992940 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b2293cf7-5721-4078-92d3-3b2f7c6b7041 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:27.025723 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:27.529677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2242203b-a05e-4375-8d77-baf73dbd6e6a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:27.529677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2242203b-a05e-4375-8d77-baf73dbd6e6a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:27.531641 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b2293cf7-5721-4078-92d3-3b2f7c6b7041 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:27.531641 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b2293cf7-5721-4078-92d3-3b2f7c6b7041 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1714 microversion: 2.1 time: 0.295303 Jun 10 13:07:27.532519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2242203b-a05e-4375-8d77-baf73dbd6e6a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:27.532519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2242203b-a05e-4375-8d77-baf73dbd6e6a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:27.532856 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 143/303] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:26 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1714 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:27.533013 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2242203b-a05e-4375-8d77-baf73dbd6e6a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:27.533013 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2242203b-a05e-4375-8d77-baf73dbd6e6a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:27.533013 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2242203b-a05e-4375-8d77-baf73dbd6e6a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:27.533635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2242203b-a05e-4375-8d77-baf73dbd6e6a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:27.533635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2242203b-a05e-4375-8d77-baf73dbd6e6a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:27.533635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2242203b-a05e-4375-8d77-baf73dbd6e6a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1738 microversion: 2.20 time: 0.273057 Jun 10 13:07:27.535628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 161/304] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:27 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1738 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:07:27.587582 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-dced04ad-20f7-4f80-9381-8adf652ae464 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:27.588591 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-dced04ad-20f7-4f80-9381-8adf652ae464 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:27.595648 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dced04ad-20f7-4f80-9381-8adf652ae464 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:27.701929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dced04ad-20f7-4f80-9381-8adf652ae464 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:27.701929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dced04ad-20f7-4f80-9381-8adf652ae464 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:27.701929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dced04ad-20f7-4f80-9381-8adf652ae464 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:27.756671 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-dced04ad-20f7-4f80-9381-8adf652ae464 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:27.766083 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-dced04ad-20f7-4f80-9381-8adf652ae464 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1313 microversion: 2.1 time: 0.184166 Jun 10 13:07:27.767072 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 144/305] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:27 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1313 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:27.913919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-89b6d25d-2d4e-4172-a0d8-84c2b68ca804 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:27.915204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-89b6d25d-2d4e-4172-a0d8-84c2b68ca804 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:27.925007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-89b6d25d-2d4e-4172-a0d8-84c2b68ca804 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:27.940871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-89b6d25d-2d4e-4172-a0d8-84c2b68ca804 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:27.943494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-89b6d25d-2d4e-4172-a0d8-84c2b68ca804 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:27.949391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-89b6d25d-2d4e-4172-a0d8-84c2b68ca804 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:28.256708 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-89b6d25d-2d4e-4172-a0d8-84c2b68ca804 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:28.285386 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-89b6d25d-2d4e-4172-a0d8-84c2b68ca804 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.372205 Jun 10 13:07:28.288003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 162/306] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:27 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:28.543878 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b3134ac2-e973-4db1-a2fd-0a3867aa0885 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:28.546856 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b3134ac2-e973-4db1-a2fd-0a3867aa0885 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:28.555999 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b3134ac2-e973-4db1-a2fd-0a3867aa0885 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:28.556438 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b3134ac2-e973-4db1-a2fd-0a3867aa0885 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:28.559914 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b3134ac2-e973-4db1-a2fd-0a3867aa0885 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:28.559914 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b3134ac2-e973-4db1-a2fd-0a3867aa0885 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:28.790990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ba793700-8055-475b-baa0-3d2a2e29a4e5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:28.792040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ba793700-8055-475b-baa0-3d2a2e29a4e5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:28.813755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba793700-8055-475b-baa0-3d2a2e29a4e5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:28.814960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba793700-8055-475b-baa0-3d2a2e29a4e5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:28.817583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba793700-8055-475b-baa0-3d2a2e29a4e5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:28.819124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba793700-8055-475b-baa0-3d2a2e29a4e5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:28.882765 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b3134ac2-e973-4db1-a2fd-0a3867aa0885 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:28.899141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b3134ac2-e973-4db1-a2fd-0a3867aa0885 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:28.905515 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b3134ac2-e973-4db1-a2fd-0a3867aa0885 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:28.908449 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b3134ac2-e973-4db1-a2fd-0a3867aa0885 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1738 microversion: 2.20 time: 0.367477 Jun 10 13:07:28.912505 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 145/307] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:28 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1738 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:28.936233 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-55a91785-f493-4617-b14d-bc319822bd18 req-6180f381-9586-4d57-b254-a51f0d4d926e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "be7b7cb1-c27c-410a-8434-271566df4aba", "name": "network-vif-plugged", "status": "completed", "tag": "09e26ef0-4ce5-4d09-bbbf-e96cf7784092"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:28.965344 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-55a91785-f493-4617-b14d-bc319822bd18 req-6180f381-9586-4d57-b254-a51f0d4d926e service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:28.967918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-55a91785-f493-4617-b14d-bc319822bd18 req-6180f381-9586-4d57-b254-a51f0d4d926e service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:28.994277 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-55a91785-f493-4617-b14d-bc319822bd18 req-6180f381-9586-4d57-b254-a51f0d4d926e service nova] Creating event network-vif-plugged:09e26ef0-4ce5-4d09-bbbf-e96cf7784092 for instance be7b7cb1-c27c-410a-8434-271566df4aba on hv-cinder-maste Jun 10 13:07:29.008045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-55a91785-f493-4617-b14d-bc319822bd18 req-6180f381-9586-4d57-b254-a51f0d4d926e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.073777 Jun 10 13:07:29.008045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 146/308] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:07:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:07:29.045112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ba793700-8055-475b-baa0-3d2a2e29a4e5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:29.076375 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ba793700-8055-475b-baa0-3d2a2e29a4e5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1313 microversion: 2.1 time: 0.287355 Jun 10 13:07:29.076375 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 163/309] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:28 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1313 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:29.306399 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e11d6a4e-c121-4d85-a2d2-00e080400bf8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:29.307511 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e11d6a4e-c121-4d85-a2d2-00e080400bf8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:29.325025 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e11d6a4e-c121-4d85-a2d2-00e080400bf8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:29.328243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e11d6a4e-c121-4d85-a2d2-00e080400bf8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:29.330260 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e11d6a4e-c121-4d85-a2d2-00e080400bf8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:29.334004 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e11d6a4e-c121-4d85-a2d2-00e080400bf8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:29.809242 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e11d6a4e-c121-4d85-a2d2-00e080400bf8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:29.818627 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e11d6a4e-c121-4d85-a2d2-00e080400bf8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.518441 Jun 10 13:07:29.822152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 147/310] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:29 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 523 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:29.933026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1b93fcea-cda9-419b-ad61-48c1814e09e8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:29.933750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1b93fcea-cda9-419b-ad61-48c1814e09e8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:29.936097 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:29.944044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b93fcea-cda9-419b-ad61-48c1814e09e8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:29.945733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b93fcea-cda9-419b-ad61-48c1814e09e8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:29.950743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b93fcea-cda9-419b-ad61-48c1814e09e8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:29.951744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b93fcea-cda9-419b-ad61-48c1814e09e8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:30.092377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-db484d23-3b58-4e5d-be21-3808907a8efa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:30.093693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-db484d23-3b58-4e5d-be21-3808907a8efa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:30.110591 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-db484d23-3b58-4e5d-be21-3808907a8efa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:30.112084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-db484d23-3b58-4e5d-be21-3808907a8efa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:30.113486 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-db484d23-3b58-4e5d-be21-3808907a8efa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:30.115907 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-db484d23-3b58-4e5d-be21-3808907a8efa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:30.254442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1b93fcea-cda9-419b-ad61-48c1814e09e8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:30.263026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1b93fcea-cda9-419b-ad61-48c1814e09e8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:30.265487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1b93fcea-cda9-419b-ad61-48c1814e09e8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:30.266662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1b93fcea-cda9-419b-ad61-48c1814e09e8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1738 microversion: 2.20 time: 0.339617 Jun 10 13:07:30.274285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 164/311] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:29 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1738 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:30.481258 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-db484d23-3b58-4e5d-be21-3808907a8efa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:30.485047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-db484d23-3b58-4e5d-be21-3808907a8efa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1313 microversion: 2.1 time: 0.397869 Jun 10 13:07:30.489272 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 148/312] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:30 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1313 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:30.835461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ad8179de-c814-43e3-849a-a3b7464c944d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b7e78895-3e22-4baa-8b73-da6fa1f7d95b", "device": "/dev/sdb"}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:30.837151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ad8179de-c814-43e3-849a-a3b7464c944d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:30.840110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c5dd39c7-ba83-4b04-8b73-cead30d935c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:30.841812 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c5dd39c7-ba83-4b04-8b73-cead30d935c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:30.844971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad8179de-c814-43e3-849a-a3b7464c944d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:30.845421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad8179de-c814-43e3-849a-a3b7464c944d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:30.846199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad8179de-c814-43e3-849a-a3b7464c944d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:30.846632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad8179de-c814-43e3-849a-a3b7464c944d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:30.849176 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c5dd39c7-ba83-4b04-8b73-cead30d935c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:30.850079 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c5dd39c7-ba83-4b04-8b73-cead30d935c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:30.851631 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c5dd39c7-ba83-4b04-8b73-cead30d935c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:30.851631 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c5dd39c7-ba83-4b04-8b73-cead30d935c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:31.053155 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c5dd39c7-ba83-4b04-8b73-cead30d935c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:31.063425 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c5dd39c7-ba83-4b04-8b73-cead30d935c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.228509 Jun 10 13:07:31.064836 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 149/313] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:30 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:31.287904 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8e5c541c-a9eb-45e9-b741-4b4529a546fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:31.288825 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8e5c541c-a9eb-45e9-b741-4b4529a546fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:31.297915 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8e5c541c-a9eb-45e9-b741-4b4529a546fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:31.298417 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8e5c541c-a9eb-45e9-b741-4b4529a546fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:31.305001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8e5c541c-a9eb-45e9-b741-4b4529a546fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:31.313240 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8e5c541c-a9eb-45e9-b741-4b4529a546fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:31.591089 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8e5c541c-a9eb-45e9-b741-4b4529a546fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:31.600253 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8e5c541c-a9eb-45e9-b741-4b4529a546fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:31.602119 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8e5c541c-a9eb-45e9-b741-4b4529a546fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:31.603419 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8e5c541c-a9eb-45e9-b741-4b4529a546fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1738 microversion: 2.20 time: 0.320528 Jun 10 13:07:31.614547 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 150/314] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:31 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1738 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:31.619460 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-60f430a4-386d-45ec-859a-e203bc1be533 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:31.620063 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-60f430a4-386d-45ec-859a-e203bc1be533 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:31.627191 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-60f430a4-386d-45ec-859a-e203bc1be533 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:31.627619 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-60f430a4-386d-45ec-859a-e203bc1be533 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:31.628392 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-60f430a4-386d-45ec-859a-e203bc1be533 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:31.628851 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-60f430a4-386d-45ec-859a-e203bc1be533 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:32.227351 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-60f430a4-386d-45ec-859a-e203bc1be533 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:32.236626 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-60f430a4-386d-45ec-859a-e203bc1be533 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1498 microversion: 2.1 time: 0.620801 Jun 10 13:07:32.237168 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 151/315] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:31 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1498 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:32.252446 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-52f459f4-d9a9-4208-aa09-8246197e9409 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:32.253094 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-52f459f4-d9a9-4208-aa09-8246197e9409 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:32.269950 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52f459f4-d9a9-4208-aa09-8246197e9409 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:32.269950 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52f459f4-d9a9-4208-aa09-8246197e9409 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:32.269950 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52f459f4-d9a9-4208-aa09-8246197e9409 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:32.270399 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52f459f4-d9a9-4208-aa09-8246197e9409 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:32.484539 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-52f459f4-d9a9-4208-aa09-8246197e9409 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:32.499428 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-52f459f4-d9a9-4208-aa09-8246197e9409 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.261121 Jun 10 13:07:32.500098 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 152/316] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:32 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:32.702907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ad8179de-c814-43e3-849a-a3b7464c944d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "POST /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.871610 Jun 10 13:07:32.704265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 165/317] 10.222.0.25 () {60 vars in 1322 bytes} [Fri Jun 10 13:07:30 2022] POST /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7/os-volume_attachments => generated 194 bytes in 1874 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:07:32.721219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f64c1845-96fc-48da-bb3a-c9d35c6ff6b3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:32.722409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f64c1845-96fc-48da-bb3a-c9d35c6ff6b3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:32.742240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f64c1845-96fc-48da-bb3a-c9d35c6ff6b3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:32.744735 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f64c1845-96fc-48da-bb3a-c9d35c6ff6b3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:32.745686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f64c1845-96fc-48da-bb3a-c9d35c6ff6b3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:32.746067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f64c1845-96fc-48da-bb3a-c9d35c6ff6b3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:32.774477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-41069a52-7bc3-45ce-90bf-c85a1a8e91b3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Action: 'create', calling method: .version_select at 0x7f93054ff700>, body: {"keypair": {"name": "tempest-TestShelveInstance-943261541", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAgWyVCixhcK3Szxmm3kxnDx4ifSn6Tq6K1LseAsxIhfVgiVsIp2rL2N0lsUBtA1TdgpGYV0CCTEdlZKVYbzg4zFBs8snpxihDUkLU+tpHTEOiB9TKW1K6WzgtA8+cPobA=="}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:32.809028 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-41069a52-7bc3-45ce-90bf-c85a1a8e91b3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Getting quotas for project 75273f09637240f3a5cc7129927765f1. Resources: {'key_pairs'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:07:32.821951 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:32.823599 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-41069a52-7bc3-45ce-90bf-c85a1a8e91b3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Getting quotas for user 0a1368d42efc4543a908a61c48ec4472 and project 75273f09637240f3a5cc7129927765f1. Resources: {'key_pairs'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:07:32.967407 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-41069a52-7bc3-45ce-90bf-c85a1a8e91b3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.455054 Jun 10 13:07:32.968643 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 153/318] 10.222.0.25 () {60 vars in 1158 bytes} [Fri Jun 10 13:07:32 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:07:33.063908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f64c1845-96fc-48da-bb3a-c9d35c6ff6b3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:33.074577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f64c1845-96fc-48da-bb3a-c9d35c6ff6b3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:33.077756 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f64c1845-96fc-48da-bb3a-c9d35c6ff6b3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:33.079440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f64c1845-96fc-48da-bb3a-c9d35c6ff6b3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1738 microversion: 2.20 time: 0.373688 Jun 10 13:07:33.086283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 166/319] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:32 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1738 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:33.519031 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8cf8fb7b-55d6-49e3-8cdf-0ffb599266d2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:33.521999 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8cf8fb7b-55d6-49e3-8cdf-0ffb599266d2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:33.538434 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8cf8fb7b-55d6-49e3-8cdf-0ffb599266d2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:33.539406 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8cf8fb7b-55d6-49e3-8cdf-0ffb599266d2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:33.541337 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8cf8fb7b-55d6-49e3-8cdf-0ffb599266d2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:33.542588 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8cf8fb7b-55d6-49e3-8cdf-0ffb599266d2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:33.703737 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-eb0b23c1-a521-4d11-a808-77e444e5f73f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:33.704226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-eb0b23c1-a521-4d11-a808-77e444e5f73f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:33.720547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eb0b23c1-a521-4d11-a808-77e444e5f73f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:33.721083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eb0b23c1-a521-4d11-a808-77e444e5f73f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:33.723118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eb0b23c1-a521-4d11-a808-77e444e5f73f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:33.723568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eb0b23c1-a521-4d11-a808-77e444e5f73f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:33.793664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-eb0b23c1-a521-4d11-a808-77e444e5f73f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:07:33.839813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:33.890271 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8cf8fb7b-55d6-49e3-8cdf-0ffb599266d2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:33.899174 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8cf8fb7b-55d6-49e3-8cdf-0ffb599266d2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.385725 Jun 10 13:07:33.900078 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 154/320] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:33 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:34.104855 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f4895c00-620a-4e1b-95d5-519ac7440927 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:34.105728 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f4895c00-620a-4e1b-95d5-519ac7440927 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:34.112235 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f4895c00-620a-4e1b-95d5-519ac7440927 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:34.112685 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f4895c00-620a-4e1b-95d5-519ac7440927 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:34.113490 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f4895c00-620a-4e1b-95d5-519ac7440927 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:34.113921 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f4895c00-620a-4e1b-95d5-519ac7440927 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:34.279874 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-eb0b23c1-a521-4d11-a808-77e444e5f73f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "DELETE /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 204 len: 0 microversion: 2.1 time: 0.579095 Jun 10 13:07:34.282457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 167/321] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:07:33 2022] DELETE /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 0 bytes in 583 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jun 10 13:07:34.292553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7b7e1683-32ca-4784-aefd-c11ce21e2a73 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:34.294727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7b7e1683-32ca-4784-aefd-c11ce21e2a73 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:34.320050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b7e1683-32ca-4784-aefd-c11ce21e2a73 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:34.320528 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b7e1683-32ca-4784-aefd-c11ce21e2a73 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:34.325264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b7e1683-32ca-4784-aefd-c11ce21e2a73 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:34.328341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b7e1683-32ca-4784-aefd-c11ce21e2a73 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:34.539921 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f4895c00-620a-4e1b-95d5-519ac7440927 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:34.551521 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f4895c00-620a-4e1b-95d5-519ac7440927 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:34.556533 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f4895c00-620a-4e1b-95d5-519ac7440927 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:34.560988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f4895c00-620a-4e1b-95d5-519ac7440927 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1738 microversion: 2.20 time: 0.457382 Jun 10 13:07:34.572899 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 155/322] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:34 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1738 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:07:34.674223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7b7e1683-32ca-4784-aefd-c11ce21e2a73 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:34.686912 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7b7e1683-32ca-4784-aefd-c11ce21e2a73 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1504 microversion: 2.1 time: 0.399476 Jun 10 13:07:34.688099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 168/323] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:34 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1504 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:34.922165 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9a124fbc-89c7-4584-9e5a-c71d42c7e22b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:34.923290 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9a124fbc-89c7-4584-9e5a-c71d42c7e22b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:34.948782 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9a124fbc-89c7-4584-9e5a-c71d42c7e22b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:34.950131 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9a124fbc-89c7-4584-9e5a-c71d42c7e22b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:34.954387 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9a124fbc-89c7-4584-9e5a-c71d42c7e22b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:34.955697 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9a124fbc-89c7-4584-9e5a-c71d42c7e22b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:35.195657 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9a124fbc-89c7-4584-9e5a-c71d42c7e22b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:35.205042 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9a124fbc-89c7-4584-9e5a-c71d42c7e22b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.290759 Jun 10 13:07:35.205710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 156/324] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:34 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:35.576990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d7950ef9-0722-4922-9b28-5274d6e8962e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:35.577610 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d7950ef9-0722-4922-9b28-5274d6e8962e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:35.635369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d7950ef9-0722-4922-9b28-5274d6e8962e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:35.635369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d7950ef9-0722-4922-9b28-5274d6e8962e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:35.635369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d7950ef9-0722-4922-9b28-5274d6e8962e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:35.635369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d7950ef9-0722-4922-9b28-5274d6e8962e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:35.704651 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8ea6e6d7-32c2-4bb0-ad59-63c496b7d40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:35.706241 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8ea6e6d7-32c2-4bb0-ad59-63c496b7d40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:35.714058 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8ea6e6d7-32c2-4bb0-ad59-63c496b7d40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:35.715397 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8ea6e6d7-32c2-4bb0-ad59-63c496b7d40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:35.716461 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8ea6e6d7-32c2-4bb0-ad59-63c496b7d40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:35.717457 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8ea6e6d7-32c2-4bb0-ad59-63c496b7d40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:35.746286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d7950ef9-0722-4922-9b28-5274d6e8962e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:35.756867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d7950ef9-0722-4922-9b28-5274d6e8962e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:35.758674 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d7950ef9-0722-4922-9b28-5274d6e8962e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:35.760081 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d7950ef9-0722-4922-9b28-5274d6e8962e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1738 microversion: 2.20 time: 0.187247 Jun 10 13:07:35.760967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 169/325] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:35 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1738 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:07:35.940473 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8ea6e6d7-32c2-4bb0-ad59-63c496b7d40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:35.948033 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8ea6e6d7-32c2-4bb0-ad59-63c496b7d40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1504 microversion: 2.1 time: 0.247323 Jun 10 13:07:35.948853 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 157/326] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:35 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1504 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:36.233025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1d615423-dac5-42c1-9b70-2075d2363d0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:36.233025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1d615423-dac5-42c1-9b70-2075d2363d0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:36.242877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d615423-dac5-42c1-9b70-2075d2363d0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:36.243329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d615423-dac5-42c1-9b70-2075d2363d0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:36.244207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d615423-dac5-42c1-9b70-2075d2363d0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:36.244952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d615423-dac5-42c1-9b70-2075d2363d0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:36.427302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1d615423-dac5-42c1-9b70-2075d2363d0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:36.435503 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1d615423-dac5-42c1-9b70-2075d2363d0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.214467 Jun 10 13:07:36.436311 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 170/327] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:36 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:36.778303 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a3f559fc-9d3e-4e28-bdea-591f5e92f7d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:36.778948 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a3f559fc-9d3e-4e28-bdea-591f5e92f7d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:36.791069 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3f559fc-9d3e-4e28-bdea-591f5e92f7d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:36.795862 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3f559fc-9d3e-4e28-bdea-591f5e92f7d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:36.800079 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3f559fc-9d3e-4e28-bdea-591f5e92f7d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:36.800754 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3f559fc-9d3e-4e28-bdea-591f5e92f7d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:36.972197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fc5436aa-e9b8-4e04-bc94-cf5466ddc48f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:36.973534 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fc5436aa-e9b8-4e04-bc94-cf5466ddc48f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:36.985038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fc5436aa-e9b8-4e04-bc94-cf5466ddc48f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:36.985993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fc5436aa-e9b8-4e04-bc94-cf5466ddc48f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:36.986990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fc5436aa-e9b8-4e04-bc94-cf5466ddc48f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:36.987583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fc5436aa-e9b8-4e04-bc94-cf5466ddc48f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:37.008628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a3f559fc-9d3e-4e28-bdea-591f5e92f7d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:37.019363 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a3f559fc-9d3e-4e28-bdea-591f5e92f7d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:37.021048 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a3f559fc-9d3e-4e28-bdea-591f5e92f7d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:37.022308 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a3f559fc-9d3e-4e28-bdea-591f5e92f7d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1738 microversion: 2.20 time: 0.247867 Jun 10 13:07:37.023146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 158/328] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:36 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1738 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:37.174291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fc5436aa-e9b8-4e04-bc94-cf5466ddc48f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:37.205340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fc5436aa-e9b8-4e04-bc94-cf5466ddc48f tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1504 microversion: 2.1 time: 0.224224 Jun 10 13:07:37.206257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 171/329] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:36 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1504 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:37.459702 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cd1bfaad-42d9-4337-a714-8f8594ba9ff6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:37.460659 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cd1bfaad-42d9-4337-a714-8f8594ba9ff6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:37.469729 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cd1bfaad-42d9-4337-a714-8f8594ba9ff6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:37.474233 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cd1bfaad-42d9-4337-a714-8f8594ba9ff6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:37.475335 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cd1bfaad-42d9-4337-a714-8f8594ba9ff6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:37.475986 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cd1bfaad-42d9-4337-a714-8f8594ba9ff6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:37.898910 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cd1bfaad-42d9-4337-a714-8f8594ba9ff6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:37.931230 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cd1bfaad-42d9-4337-a714-8f8594ba9ff6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.477264 Jun 10 13:07:37.931969 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 159/330] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:37 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:38.041881 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-852f3b0d-4cc3-4402-82cd-16a2f0610ae7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:38.042616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-852f3b0d-4cc3-4402-82cd-16a2f0610ae7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:38.053156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-852f3b0d-4cc3-4402-82cd-16a2f0610ae7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:38.053605 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-852f3b0d-4cc3-4402-82cd-16a2f0610ae7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:38.054386 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-852f3b0d-4cc3-4402-82cd-16a2f0610ae7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:38.054821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-852f3b0d-4cc3-4402-82cd-16a2f0610ae7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:38.223228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-18be7bb9-5ca2-44d4-b806-df6596d4ad2e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:38.224153 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-18be7bb9-5ca2-44d4-b806-df6596d4ad2e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:38.231831 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-18be7bb9-5ca2-44d4-b806-df6596d4ad2e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:38.232555 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-18be7bb9-5ca2-44d4-b806-df6596d4ad2e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:38.233531 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-18be7bb9-5ca2-44d4-b806-df6596d4ad2e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:38.234143 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-18be7bb9-5ca2-44d4-b806-df6596d4ad2e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:38.367208 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-852f3b0d-4cc3-4402-82cd-16a2f0610ae7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:38.381894 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-852f3b0d-4cc3-4402-82cd-16a2f0610ae7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:38.384230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-852f3b0d-4cc3-4402-82cd-16a2f0610ae7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:38.390269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-852f3b0d-4cc3-4402-82cd-16a2f0610ae7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1738 microversion: 2.20 time: 0.352172 Jun 10 13:07:38.391428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 172/331] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:38 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1738 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:38.414440 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-18be7bb9-5ca2-44d4-b806-df6596d4ad2e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:38.435596 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-18be7bb9-5ca2-44d4-b806-df6596d4ad2e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1462 microversion: 2.1 time: 0.214405 Jun 10 13:07:38.440687 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 160/332] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:38 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1462 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:38.454449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-01f7a052-6237-4fe8-83cc-f98b49df93ac req-f8199a4d-6dc1-4416-8193-b51a1f3678c7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "be7b7cb1-c27c-410a-8434-271566df4aba", "name": "network-vif-deleted", "tag": "09e26ef0-4ce5-4d09-bbbf-e96cf7784092"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:38.465449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-01f7a052-6237-4fe8-83cc-f98b49df93ac req-f8199a4d-6dc1-4416-8193-b51a1f3678c7 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:38.466015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-01f7a052-6237-4fe8-83cc-f98b49df93ac req-f8199a4d-6dc1-4416-8193-b51a1f3678c7 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:38.479130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-01f7a052-6237-4fe8-83cc-f98b49df93ac req-f8199a4d-6dc1-4416-8193-b51a1f3678c7 service nova] Creating event network-vif-deleted:09e26ef0-4ce5-4d09-bbbf-e96cf7784092 for instance be7b7cb1-c27c-410a-8434-271566df4aba on hv-cinder-maste Jun 10 13:07:38.487372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-01f7a052-6237-4fe8-83cc-f98b49df93ac req-f8199a4d-6dc1-4416-8193-b51a1f3678c7 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050271 Jun 10 13:07:38.495071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 173/333] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:07:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:07:38.951768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5c342fc0-4e37-42db-8441-b242bac6811f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:38.952543 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5c342fc0-4e37-42db-8441-b242bac6811f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:38.958868 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5c342fc0-4e37-42db-8441-b242bac6811f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:38.959628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5c342fc0-4e37-42db-8441-b242bac6811f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:38.960610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5c342fc0-4e37-42db-8441-b242bac6811f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:38.961063 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5c342fc0-4e37-42db-8441-b242bac6811f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:38.987546 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:39.180134 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5c342fc0-4e37-42db-8441-b242bac6811f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:39.188800 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5c342fc0-4e37-42db-8441-b242bac6811f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1522 microversion: 2.1 time: 0.240771 Jun 10 13:07:39.201104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 161/334] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:38 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1522 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:39.415849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9d36e47e-23a8-4c14-ba2e-cb18e48fd770 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:39.416558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9d36e47e-23a8-4c14-ba2e-cb18e48fd770 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:39.423419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9d36e47e-23a8-4c14-ba2e-cb18e48fd770 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:39.423868 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9d36e47e-23a8-4c14-ba2e-cb18e48fd770 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:39.424652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9d36e47e-23a8-4c14-ba2e-cb18e48fd770 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:39.425297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9d36e47e-23a8-4c14-ba2e-cb18e48fd770 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:39.481922 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-29ebcf15-d564-417f-9b9b-a7d5ea40f40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:39.482857 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-29ebcf15-d564-417f-9b9b-a7d5ea40f40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:39.518604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29ebcf15-d564-417f-9b9b-a7d5ea40f40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:39.520590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29ebcf15-d564-417f-9b9b-a7d5ea40f40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:39.521512 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29ebcf15-d564-417f-9b9b-a7d5ea40f40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:39.522316 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29ebcf15-d564-417f-9b9b-a7d5ea40f40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:39.594516 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29ebcf15-d564-417f-9b9b-a7d5ea40f40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:39.594516 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29ebcf15-d564-417f-9b9b-a7d5ea40f40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:39.594516 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-29ebcf15-d564-417f-9b9b-a7d5ea40f40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lazy-loading 'fault' on Instance uuid be7b7cb1-c27c-410a-8434-271566df4aba {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:07:39.632793 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-29ebcf15-d564-417f-9b9b-a7d5ea40f40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:39.636969 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-29ebcf15-d564-417f-9b9b-a7d5ea40f40b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1303 microversion: 2.1 time: 0.176317 Jun 10 13:07:39.639912 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 162/335] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:39 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1303 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:39.680129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9d36e47e-23a8-4c14-ba2e-cb18e48fd770 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:39.688709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9d36e47e-23a8-4c14-ba2e-cb18e48fd770 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:39.690235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9d36e47e-23a8-4c14-ba2e-cb18e48fd770 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:39.691416 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9d36e47e-23a8-4c14-ba2e-cb18e48fd770 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1738 microversion: 2.20 time: 0.283154 Jun 10 13:07:39.691982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 174/336] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:39 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1738 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:40.205867 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-574289ac-517d-46e8-8649-e34773a051e3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:40.206480 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-574289ac-517d-46e8-8649-e34773a051e3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:40.213055 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-574289ac-517d-46e8-8649-e34773a051e3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:40.213509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-574289ac-517d-46e8-8649-e34773a051e3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:40.214268 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-574289ac-517d-46e8-8649-e34773a051e3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:40.214701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-574289ac-517d-46e8-8649-e34773a051e3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:40.364051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-574289ac-517d-46e8-8649-e34773a051e3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:40.373172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-574289ac-517d-46e8-8649-e34773a051e3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1502 microversion: 2.1 time: 0.170604 Jun 10 13:07:40.373750 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 163/337] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:40 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1502 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:40.391838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4b56f41e-95ea-4d94-b44e-83188b7a8ca9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:40.392794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4b56f41e-95ea-4d94-b44e-83188b7a8ca9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:40.402732 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b56f41e-95ea-4d94-b44e-83188b7a8ca9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:40.403377 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b56f41e-95ea-4d94-b44e-83188b7a8ca9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:40.404275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b56f41e-95ea-4d94-b44e-83188b7a8ca9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:40.404802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b56f41e-95ea-4d94-b44e-83188b7a8ca9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:40.636278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4b56f41e-95ea-4d94-b44e-83188b7a8ca9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:40.647813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4b56f41e-95ea-4d94-b44e-83188b7a8ca9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1515 microversion: 2.1 time: 0.260790 Jun 10 13:07:40.648464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 175/338] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:40 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1515 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:40.656111 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-739e7280-9f8f-4efc-90f3-8042241676e0 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:40.661892 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-739e7280-9f8f-4efc-90f3-8042241676e0 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:40.677400 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-739e7280-9f8f-4efc-90f3-8042241676e0 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:40.677400 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-739e7280-9f8f-4efc-90f3-8042241676e0 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:40.677400 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-739e7280-9f8f-4efc-90f3-8042241676e0 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:40.677400 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-739e7280-9f8f-4efc-90f3-8042241676e0 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:40.709949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-aa0bea81-7764-4c7d-a20d-41bd6462c4aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:40.711486 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-aa0bea81-7764-4c7d-a20d-41bd6462c4aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:40.724810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aa0bea81-7764-4c7d-a20d-41bd6462c4aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:40.724810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aa0bea81-7764-4c7d-a20d-41bd6462c4aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:40.724810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aa0bea81-7764-4c7d-a20d-41bd6462c4aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:40.725779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aa0bea81-7764-4c7d-a20d-41bd6462c4aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:40.739312 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-739e7280-9f8f-4efc-90f3-8042241676e0 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:40.739903 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-739e7280-9f8f-4efc-90f3-8042241676e0 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:40.742265 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-739e7280-9f8f-4efc-90f3-8042241676e0 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lazy-loading 'fault' on Instance uuid be7b7cb1-c27c-410a-8434-271566df4aba {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:07:40.788249 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-739e7280-9f8f-4efc-90f3-8042241676e0 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:40.796746 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-739e7280-9f8f-4efc-90f3-8042241676e0 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 200 len: 1303 microversion: 2.1 time: 0.144459 Jun 10 13:07:40.807316 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 164/339] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:40 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 1303 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:40.962617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-aa0bea81-7764-4c7d-a20d-41bd6462c4aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:40.973418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-aa0bea81-7764-4c7d-a20d-41bd6462c4aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:40.978248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-aa0bea81-7764-4c7d-a20d-41bd6462c4aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:40.978248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-aa0bea81-7764-4c7d-a20d-41bd6462c4aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1738 microversion: 2.20 time: 0.270440 Jun 10 13:07:40.978248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 176/340] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:40 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1738 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:41.817179 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-99c79bb4-f67a-4641-ae98-3c0bed15eca5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:41.817794 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-99c79bb4-f67a-4641-ae98-3c0bed15eca5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: be7b7cb1-c27c-410a-8434-271566df4aba] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:41.823768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-99c79bb4-f67a-4641-ae98-3c0bed15eca5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:41.826390 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-99c79bb4-f67a-4641-ae98-3c0bed15eca5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:41.826390 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-99c79bb4-f67a-4641-ae98-3c0bed15eca5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:41.826390 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-99c79bb4-f67a-4641-ae98-3c0bed15eca5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:41.838891 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.wsgi [None req-99c79bb4-f67a-4641-ae98-3c0bed15eca5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] HTTP exception thrown: Instance be7b7cb1-c27c-410a-8434-271566df4aba could not be found. Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-99c79bb4-f67a-4641-ae98-3c0bed15eca5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Request method failure captured: Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: request: GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba HTTP/1.1 Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept: application/json Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept-Encoding: identity Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Connection: close Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Length: 0 Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Type: application/json Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Host: 10.222.0.25 Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Openstack-System-Scope: None Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: User-Agent: python-urllib3/1.26.9 Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Auth-Token: gAAAAABio0H689WBWNCTm_Kgo3HOHebBiL8zdvrAVTwjaBaT7O7WxeG7Ru0F6ZlnGiauVQNyjr5U3uKPwYpJucZR8wjbqxuJJNUcu4F7JleSQU0vA2UIHtWi1ZlyReVm_U2_mNKXVDR8kx0ZvsbFDyXEyOd0fTBeIkt6KVXqnZug71yGsoqBV10 Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Id: None Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Name: None Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Identity-Status: Confirmed Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Is-Admin-Project: True Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Id: default Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Name: Default Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Id: 0fdf6af645ba4a02a6a7278eb1fdff0d Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Name: tempest-VolumesActionsTest-1522136130 Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Role: reader,member Jun 10 13:07:41.840526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Roles: reader,member Jun 10 13:07:41.842587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/0fdf6af645ba4a02a6a7278eb1fdff0d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0fdf6af645ba4a02a6a7278eb1fdff0d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_0fdf6af645ba4a02a6a7278eb1fdff0d", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0fdf6af645ba4a02a6a7278eb1fdff0d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:07:41.842587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant: tempest-VolumesActionsTest-1522136130 Jun 10 13:07:41.842587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Id: 0fdf6af645ba4a02a6a7278eb1fdff0d Jun 10 13:07:41.842587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Name: tempest-VolumesActionsTest-1522136130 Jun 10 13:07:41.842587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User: tempest-VolumesActionsTest-1522136130-project Jun 10 13:07:41.842587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Id: default Jun 10 13:07:41.842587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Name: Default Jun 10 13:07:41.842587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Id: 24343309fb924981a37899f56f15b8ba Jun 10 13:07:41.842587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Name: tempest-VolumesActionsTest-1522136130-project Jun 10 13:07:41.842587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: method: > Jun 10 13:07:41.842587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: exception: Instance be7b7cb1-c27c-410a-8434-271566df4aba could not be found. Jun 10 13:07:41.842587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: action_args: {'id': 'be7b7cb1-c27c-410a-8434-271566df4aba'} Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance be7b7cb1-c27c-410a-8434-271566df4aba could not be found. Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:07:41.843839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance be7b7cb1-c27c-410a-8434-271566df4aba could not be found. Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance be7b7cb1-c27c-410a-8434-271566df4aba could not be found. Jun 10 13:07:41.845781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:07:41.846832 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-99c79bb4-f67a-4641-ae98-3c0bed15eca5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Returning 404 to user: Instance be7b7cb1-c27c-410a-8434-271566df4aba could not be found. {{(pid=118093) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:07:41.846832 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-99c79bb4-f67a-4641-ae98-3c0bed15eca5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba" status: 404 len: 111 microversion: 2.1 time: 0.032591 Jun 10 13:07:41.846832 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 165/341] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:41 2022] GET /compute/v2.1/servers/be7b7cb1-c27c-410a-8434-271566df4aba => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:07:41.947837 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-390580817", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "networks": [{"uuid": "1965e685-7775-47a2-a72b-2fe332a3bed7"}]}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:41.995807 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d5909549-49e6-42c2-a13d-4d8b07a78b93 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:41.996695 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d5909549-49e6-42c2-a13d-4d8b07a78b93 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:42.002388 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.network.neutron [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] validate_networks() for [('1965e685-7775-47a2-a72b-2fe332a3bed7', None, None, None, None, None)] {{(pid=118092) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:07:42.008620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d5909549-49e6-42c2-a13d-4d8b07a78b93 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:42.008620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d5909549-49e6-42c2-a13d-4d8b07a78b93 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:42.008620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d5909549-49e6-42c2-a13d-4d8b07a78b93 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:42.008620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d5909549-49e6-42c2-a13d-4d8b07a78b93 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:42.152414 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d5909549-49e6-42c2-a13d-4d8b07a78b93 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:42.161821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d5909549-49e6-42c2-a13d-4d8b07a78b93 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:42.164122 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d5909549-49e6-42c2-a13d-4d8b07a78b93 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:42.165389 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d5909549-49e6-42c2-a13d-4d8b07a78b93 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1738 microversion: 2.20 time: 0.173842 Jun 10 13:07:42.166259 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 166/342] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:41 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1738 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:42.397681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Flavor limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:07:42.397681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Image limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:07:42.397681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Flavor pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:07:42.397681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Image pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:07:42.397681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:07:42.406300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:42.406750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:42.407598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:42.408032 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:42.425381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Getting quotas for project 0fdf6af645ba4a02a6a7278eb1fdff0d. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:07:42.429664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Getting quotas for user 24343309fb924981a37899f56f15b8ba and project 0fdf6af645ba4a02a6a7278eb1fdff0d. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:07:42.441783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Going to run 1 instances... {{(pid=118092) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:07:42.444106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118092) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:07:42.618641 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-59bdc84b-8943-4550-8bcf-008b6dbf0938 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:42.619865 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-59bdc84b-8943-4550-8bcf-008b6dbf0938 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:42.626558 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59bdc84b-8943-4550-8bcf-008b6dbf0938 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:42.627014 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59bdc84b-8943-4550-8bcf-008b6dbf0938 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:42.627779 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59bdc84b-8943-4550-8bcf-008b6dbf0938 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:42.628428 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59bdc84b-8943-4550-8bcf-008b6dbf0938 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:42.630429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f0cdcf91-3902-45a2-9fd3-146a3f24c5c4 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.686343 Jun 10 13:07:42.631432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 177/343] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:07:41 2022] POST /compute/v2.1/servers => generated 380 bytes in 688 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jun 10 13:07:42.643906 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0a9a876b-d5fd-44fc-b088-7b68017d8c1e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:42.644934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0a9a876b-d5fd-44fc-b088-7b68017d8c1e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:42.655578 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-59bdc84b-8943-4550-8bcf-008b6dbf0938 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lazy-loading 'flavor' on Instance uuid 5d205551-9f23-46f8-b67b-ca5bafd58182 {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:07:42.729635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0a9a876b-d5fd-44fc-b088-7b68017d8c1e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:42.737851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0a9a876b-d5fd-44fc-b088-7b68017d8c1e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1213 microversion: 2.1 time: 0.100165 Jun 10 13:07:42.751225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 178/344] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:42 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1213 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:42.850180 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=4 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:43.062986 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-59bdc84b-8943-4550-8bcf-008b6dbf0938 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "POST /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182/action" status: 200 len: 32 microversion: 2.1 time: 0.447753 Jun 10 13:07:43.064943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 167/345] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Jun 10 13:07:42 2022] POST /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182/action => generated 32 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jun 10 13:07:43.074566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-72455ee9-f0f0-42bf-a4ea-11133554744b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:43.075248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-72455ee9-f0f0-42bf-a4ea-11133554744b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:43.082558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72455ee9-f0f0-42bf-a4ea-11133554744b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:43.083094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72455ee9-f0f0-42bf-a4ea-11133554744b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:43.083872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72455ee9-f0f0-42bf-a4ea-11133554744b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:43.084310 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72455ee9-f0f0-42bf-a4ea-11133554744b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:43.184663 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-77cf8ebb-1740-4ca3-b48c-f04be3ac6ad9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:43.186321 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-77cf8ebb-1740-4ca3-b48c-f04be3ac6ad9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:43.193667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-77cf8ebb-1740-4ca3-b48c-f04be3ac6ad9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:43.196725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-77cf8ebb-1740-4ca3-b48c-f04be3ac6ad9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:43.196725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-77cf8ebb-1740-4ca3-b48c-f04be3ac6ad9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:43.196725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-77cf8ebb-1740-4ca3-b48c-f04be3ac6ad9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:43.233103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-72455ee9-f0f0-42bf-a4ea-11133554744b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:43.241518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-72455ee9-f0f0-42bf-a4ea-11133554744b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.171658 Jun 10 13:07:43.242219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 179/346] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:43 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:43.413964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-77cf8ebb-1740-4ca3-b48c-f04be3ac6ad9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:43.427906 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-77cf8ebb-1740-4ca3-b48c-f04be3ac6ad9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:43.429555 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-77cf8ebb-1740-4ca3-b48c-f04be3ac6ad9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:43.436485 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-77cf8ebb-1740-4ca3-b48c-f04be3ac6ad9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1720 microversion: 2.20 time: 0.253909 Jun 10 13:07:43.437286 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 168/347] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:43 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1720 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:07:43.755400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6eb6d59b-da6e-4a80-a603-be25b027a68e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:43.756014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6eb6d59b-da6e-4a80-a603-be25b027a68e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:43.762628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6eb6d59b-da6e-4a80-a603-be25b027a68e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:43.763198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6eb6d59b-da6e-4a80-a603-be25b027a68e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:43.764557 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6eb6d59b-da6e-4a80-a603-be25b027a68e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:43.765100 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6eb6d59b-da6e-4a80-a603-be25b027a68e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:43.829050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6eb6d59b-da6e-4a80-a603-be25b027a68e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:43.837377 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6eb6d59b-da6e-4a80-a603-be25b027a68e tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1217 microversion: 2.1 time: 0.086016 Jun 10 13:07:43.838126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 180/348] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:43 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1217 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:44.259445 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e8bc89cd-ebf8-4499-9b42-da767b638db3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:44.260050 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e8bc89cd-ebf8-4499-9b42-da767b638db3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:44.267353 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e8bc89cd-ebf8-4499-9b42-da767b638db3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:44.267792 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e8bc89cd-ebf8-4499-9b42-da767b638db3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:44.268562 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e8bc89cd-ebf8-4499-9b42-da767b638db3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:44.269020 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e8bc89cd-ebf8-4499-9b42-da767b638db3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:44.435501 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e8bc89cd-ebf8-4499-9b42-da767b638db3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:44.443075 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e8bc89cd-ebf8-4499-9b42-da767b638db3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.187445 Jun 10 13:07:44.443622 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 169/349] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:44 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:44.454673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5d120429-ddaa-497e-8b5e-f68ca455f052 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:44.455278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5d120429-ddaa-497e-8b5e-f68ca455f052 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:44.461807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d120429-ddaa-497e-8b5e-f68ca455f052 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:44.462260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d120429-ddaa-497e-8b5e-f68ca455f052 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:44.463026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d120429-ddaa-497e-8b5e-f68ca455f052 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:44.463492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d120429-ddaa-497e-8b5e-f68ca455f052 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:44.629029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5d120429-ddaa-497e-8b5e-f68ca455f052 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:44.639813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5d120429-ddaa-497e-8b5e-f68ca455f052 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:44.641455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5d120429-ddaa-497e-8b5e-f68ca455f052 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:44.642836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5d120429-ddaa-497e-8b5e-f68ca455f052 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1720 microversion: 2.20 time: 0.192051 Jun 10 13:07:44.644908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 181/350] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:44 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1720 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:44.855697 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8f805582-fb95-4f27-88ba-4c3e705f1734 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:44.856301 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8f805582-fb95-4f27-88ba-4c3e705f1734 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:44.862674 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8f805582-fb95-4f27-88ba-4c3e705f1734 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:44.863123 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8f805582-fb95-4f27-88ba-4c3e705f1734 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:44.863879 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8f805582-fb95-4f27-88ba-4c3e705f1734 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:44.864310 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8f805582-fb95-4f27-88ba-4c3e705f1734 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:44.923466 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8f805582-fb95-4f27-88ba-4c3e705f1734 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:44.930895 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8f805582-fb95-4f27-88ba-4c3e705f1734 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1209 microversion: 2.1 time: 0.079177 Jun 10 13:07:44.931447 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 170/351] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:44 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1209 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:45.461858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c3e3ee48-1372-477a-bd05-24d331ca4300 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:45.462590 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c3e3ee48-1372-477a-bd05-24d331ca4300 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:45.469955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3e3ee48-1372-477a-bd05-24d331ca4300 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:45.470406 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3e3ee48-1372-477a-bd05-24d331ca4300 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:45.471178 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3e3ee48-1372-477a-bd05-24d331ca4300 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:45.471614 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3e3ee48-1372-477a-bd05-24d331ca4300 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:45.604890 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c3e3ee48-1372-477a-bd05-24d331ca4300 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:45.612613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c3e3ee48-1372-477a-bd05-24d331ca4300 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.155250 Jun 10 13:07:45.613213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 182/352] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:45 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:45.666891 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-19052185-9a9f-4f86-b942-7493902330be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:45.667492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-19052185-9a9f-4f86-b942-7493902330be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:45.674224 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-19052185-9a9f-4f86-b942-7493902330be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:45.674674 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-19052185-9a9f-4f86-b942-7493902330be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:45.675437 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-19052185-9a9f-4f86-b942-7493902330be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:45.675868 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-19052185-9a9f-4f86-b942-7493902330be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:45.804320 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-19052185-9a9f-4f86-b942-7493902330be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:45.812711 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-19052185-9a9f-4f86-b942-7493902330be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:45.814247 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-19052185-9a9f-4f86-b942-7493902330be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:45.815402 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-19052185-9a9f-4f86-b942-7493902330be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1720 microversion: 2.20 time: 0.152398 Jun 10 13:07:45.815959 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 171/353] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:45 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1720 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:45.948029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e514d5d5-dc3d-4e46-a599-92ef109a9291 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:45.948639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e514d5d5-dc3d-4e46-a599-92ef109a9291 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:45.955308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e514d5d5-dc3d-4e46-a599-92ef109a9291 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:45.955759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e514d5d5-dc3d-4e46-a599-92ef109a9291 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:45.956521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e514d5d5-dc3d-4e46-a599-92ef109a9291 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:45.957010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e514d5d5-dc3d-4e46-a599-92ef109a9291 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:46.016149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e514d5d5-dc3d-4e46-a599-92ef109a9291 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:46.023508 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e514d5d5-dc3d-4e46-a599-92ef109a9291 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1209 microversion: 2.1 time: 0.079327 Jun 10 13:07:46.024076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 183/354] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:45 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1209 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:46.629812 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5cd887bb-8769-4348-bdd4-caacf5c0da63 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:46.630417 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5cd887bb-8769-4348-bdd4-caacf5c0da63 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:46.636758 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5cd887bb-8769-4348-bdd4-caacf5c0da63 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:46.638033 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5cd887bb-8769-4348-bdd4-caacf5c0da63 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:46.638033 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5cd887bb-8769-4348-bdd4-caacf5c0da63 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:46.639108 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5cd887bb-8769-4348-bdd4-caacf5c0da63 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:46.779690 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5cd887bb-8769-4348-bdd4-caacf5c0da63 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:46.787104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5cd887bb-8769-4348-bdd4-caacf5c0da63 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.161694 Jun 10 13:07:46.787656 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 172/355] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:46 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:46.835159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-373346f4-0b0a-4cc3-a93d-4e0f46e7acc0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:46.835764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-373346f4-0b0a-4cc3-a93d-4e0f46e7acc0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:46.843568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-373346f4-0b0a-4cc3-a93d-4e0f46e7acc0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:46.844017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-373346f4-0b0a-4cc3-a93d-4e0f46e7acc0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:46.844809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-373346f4-0b0a-4cc3-a93d-4e0f46e7acc0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:46.845249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-373346f4-0b0a-4cc3-a93d-4e0f46e7acc0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:46.975663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-373346f4-0b0a-4cc3-a93d-4e0f46e7acc0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:46.984179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-373346f4-0b0a-4cc3-a93d-4e0f46e7acc0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:46.985970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-373346f4-0b0a-4cc3-a93d-4e0f46e7acc0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:46.987114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-373346f4-0b0a-4cc3-a93d-4e0f46e7acc0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1720 microversion: 2.20 time: 0.156746 Jun 10 13:07:46.987666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 184/356] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:46 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1720 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:47.043163 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ae4756c2-ddb5-41e6-99c1-8feff57ea235 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:47.043758 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ae4756c2-ddb5-41e6-99c1-8feff57ea235 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:47.050036 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ae4756c2-ddb5-41e6-99c1-8feff57ea235 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:47.054418 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ae4756c2-ddb5-41e6-99c1-8feff57ea235 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:47.054418 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ae4756c2-ddb5-41e6-99c1-8feff57ea235 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:47.054418 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ae4756c2-ddb5-41e6-99c1-8feff57ea235 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:47.112182 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ae4756c2-ddb5-41e6-99c1-8feff57ea235 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:47.119559 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ae4756c2-ddb5-41e6-99c1-8feff57ea235 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1209 microversion: 2.1 time: 0.080309 Jun 10 13:07:47.120120 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 173/357] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:47 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1209 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:47.804713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7324ce99-a38a-4d76-ab05-11e31264d9ce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:47.805350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7324ce99-a38a-4d76-ab05-11e31264d9ce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:47.812418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7324ce99-a38a-4d76-ab05-11e31264d9ce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:47.812903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7324ce99-a38a-4d76-ab05-11e31264d9ce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:47.813699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7324ce99-a38a-4d76-ab05-11e31264d9ce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:47.814125 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7324ce99-a38a-4d76-ab05-11e31264d9ce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:47.830919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:47.942716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7324ce99-a38a-4d76-ab05-11e31264d9ce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:47.949929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7324ce99-a38a-4d76-ab05-11e31264d9ce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.149194 Jun 10 13:07:47.950484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 185/358] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:47 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:48.004663 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8906d428-54c2-4ab3-b3b1-835bd8d29d74 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:48.005303 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8906d428-54c2-4ab3-b3b1-835bd8d29d74 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:48.011807 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8906d428-54c2-4ab3-b3b1-835bd8d29d74 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:48.012255 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8906d428-54c2-4ab3-b3b1-835bd8d29d74 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:48.013020 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8906d428-54c2-4ab3-b3b1-835bd8d29d74 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:48.013522 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8906d428-54c2-4ab3-b3b1-835bd8d29d74 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:48.136860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2ffa4731-61c2-4c4a-820a-7efb0069c390 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:48.137563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2ffa4731-61c2-4c4a-820a-7efb0069c390 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:48.144791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ffa4731-61c2-4c4a-820a-7efb0069c390 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:48.145227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ffa4731-61c2-4c4a-820a-7efb0069c390 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:48.147634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ffa4731-61c2-4c4a-820a-7efb0069c390 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:48.148067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ffa4731-61c2-4c4a-820a-7efb0069c390 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:48.155435 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8906d428-54c2-4ab3-b3b1-835bd8d29d74 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:48.163990 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8906d428-54c2-4ab3-b3b1-835bd8d29d74 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:48.165842 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8906d428-54c2-4ab3-b3b1-835bd8d29d74 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:48.167423 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8906d428-54c2-4ab3-b3b1-835bd8d29d74 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1720 microversion: 2.20 time: 0.166576 Jun 10 13:07:48.168185 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 174/359] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:47 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1720 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:48.208910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2ffa4731-61c2-4c4a-820a-7efb0069c390 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:48.216190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2ffa4731-61c2-4c4a-820a-7efb0069c390 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1209 microversion: 2.1 time: 0.083293 Jun 10 13:07:48.217284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 186/360] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:48 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1209 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:48.265529 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-36bcc432-bff2-4d0e-bcea-92c073a6b8dd req-fbd8e373-bc2f-444c-830a-c1ee67856379 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "db56898a-dff9-4cad-bf8d-f84b53247fc3", "name": "network-vif-unplugged", "status": "completed", "tag": "31e1a9d0-3f1e-4221-a431-639f0e4a6ff4"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:48.273459 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-36bcc432-bff2-4d0e-bcea-92c073a6b8dd req-fbd8e373-bc2f-444c-830a-c1ee67856379 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:48.273891 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-36bcc432-bff2-4d0e-bcea-92c073a6b8dd req-fbd8e373-bc2f-444c-830a-c1ee67856379 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:48.284996 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-36bcc432-bff2-4d0e-bcea-92c073a6b8dd req-fbd8e373-bc2f-444c-830a-c1ee67856379 service nova] Creating event network-vif-unplugged:31e1a9d0-3f1e-4221-a431-639f0e4a6ff4 for instance db56898a-dff9-4cad-bf8d-f84b53247fc3 on hv-cinder-maste Jun 10 13:07:48.289938 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-36bcc432-bff2-4d0e-bcea-92c073a6b8dd req-fbd8e373-bc2f-444c-830a-c1ee67856379 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.028651 Jun 10 13:07:48.290508 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 175/361] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:07:48 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:07:48.968968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7ac74d82-7218-4550-a37d-115b7f8fc28f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:48.969599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7ac74d82-7218-4550-a37d-115b7f8fc28f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:48.976510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ac74d82-7218-4550-a37d-115b7f8fc28f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:48.977004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ac74d82-7218-4550-a37d-115b7f8fc28f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:48.977786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ac74d82-7218-4550-a37d-115b7f8fc28f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:48.978223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ac74d82-7218-4550-a37d-115b7f8fc28f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:49.112266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7ac74d82-7218-4550-a37d-115b7f8fc28f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:49.120287 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7ac74d82-7218-4550-a37d-115b7f8fc28f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.156475 Jun 10 13:07:49.120892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 187/362] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:48 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:49.190373 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f50ed194-ea6e-4da4-9140-818e247da410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:49.191140 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f50ed194-ea6e-4da4-9140-818e247da410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:49.197462 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f50ed194-ea6e-4da4-9140-818e247da410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:49.197904 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f50ed194-ea6e-4da4-9140-818e247da410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:49.198659 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f50ed194-ea6e-4da4-9140-818e247da410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:49.199091 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f50ed194-ea6e-4da4-9140-818e247da410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:49.233719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b3cc18ed-f8bd-48db-b3e7-fdf37a499d97 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:49.235224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b3cc18ed-f8bd-48db-b3e7-fdf37a499d97 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:49.247536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3cc18ed-f8bd-48db-b3e7-fdf37a499d97 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:49.247536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3cc18ed-f8bd-48db-b3e7-fdf37a499d97 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:49.248899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3cc18ed-f8bd-48db-b3e7-fdf37a499d97 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:49.250201 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3cc18ed-f8bd-48db-b3e7-fdf37a499d97 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:49.337473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b3cc18ed-f8bd-48db-b3e7-fdf37a499d97 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:49.347606 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b3cc18ed-f8bd-48db-b3e7-fdf37a499d97 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1265 microversion: 2.1 time: 0.117828 Jun 10 13:07:49.348269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 188/363] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:49 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1265 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:49.371223 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f50ed194-ea6e-4da4-9140-818e247da410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:49.374192 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:49.380466 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f50ed194-ea6e-4da4-9140-818e247da410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:49.382033 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f50ed194-ea6e-4da4-9140-818e247da410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:49.383223 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f50ed194-ea6e-4da4-9140-818e247da410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1723 microversion: 2.20 time: 0.197861 Jun 10 13:07:49.383775 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 176/364] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:49 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1723 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:49.401711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-35249634-d3f1-4f2f-8b67-0ecfb051f192 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "92fca8f5-5073-4d37-8ddf-ad6b329c0486"}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:49.402976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-35249634-d3f1-4f2f-8b67-0ecfb051f192 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:49.409711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35249634-d3f1-4f2f-8b67-0ecfb051f192 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:49.410160 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35249634-d3f1-4f2f-8b67-0ecfb051f192 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:49.411115 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35249634-d3f1-4f2f-8b67-0ecfb051f192 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:49.411548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35249634-d3f1-4f2f-8b67-0ecfb051f192 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:50.138172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3db9aa0e-72e9-4371-b2eb-81a690cf0b8e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:50.138822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3db9aa0e-72e9-4371-b2eb-81a690cf0b8e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:50.145540 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3db9aa0e-72e9-4371-b2eb-81a690cf0b8e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:50.145987 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3db9aa0e-72e9-4371-b2eb-81a690cf0b8e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:50.146774 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3db9aa0e-72e9-4371-b2eb-81a690cf0b8e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:50.147205 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3db9aa0e-72e9-4371-b2eb-81a690cf0b8e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:50.290691 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3db9aa0e-72e9-4371-b2eb-81a690cf0b8e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:50.299260 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3db9aa0e-72e9-4371-b2eb-81a690cf0b8e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.164913 Jun 10 13:07:50.299822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 177/365] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:50 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:50.364686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-35249634-d3f1-4f2f-8b67-0ecfb051f192 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "POST /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.967176 Jun 10 13:07:50.365303 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-02c1cfa2-11a0-44c0-96d0-1158a0ca38ff tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:50.365442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 189/366] 10.222.0.25 () {62 vars in 1363 bytes} [Fri Jun 10 13:07:49 2022] POST /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 188 bytes in 969 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:07:50.365805 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-02c1cfa2-11a0-44c0-96d0-1158a0ca38ff tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:50.371955 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-02c1cfa2-11a0-44c0-96d0-1158a0ca38ff tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:50.372403 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-02c1cfa2-11a0-44c0-96d0-1158a0ca38ff tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:50.373181 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-02c1cfa2-11a0-44c0-96d0-1158a0ca38ff tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:50.373611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-02c1cfa2-11a0-44c0-96d0-1158a0ca38ff tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:50.435155 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-02c1cfa2-11a0-44c0-96d0-1158a0ca38ff tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:50.442574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-02c1cfa2-11a0-44c0-96d0-1158a0ca38ff tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1273 microversion: 2.1 time: 0.081949 Jun 10 13:07:50.443135 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 178/367] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:50 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1273 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:50.489498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a0ebdbf7-7308-4bc6-8a88-4a3d7374cac6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:50.490049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a0ebdbf7-7308-4bc6-8a88-4a3d7374cac6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:50.496839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0ebdbf7-7308-4bc6-8a88-4a3d7374cac6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:50.497293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0ebdbf7-7308-4bc6-8a88-4a3d7374cac6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:50.498079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0ebdbf7-7308-4bc6-8a88-4a3d7374cac6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:50.498512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0ebdbf7-7308-4bc6-8a88-4a3d7374cac6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:50.715385 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a0ebdbf7-7308-4bc6-8a88-4a3d7374cac6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "POST /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/action" status: 202 len: 0 microversion: 2.20 time: 0.229965 Jun 10 13:07:50.716137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 190/368] 10.222.0.25 () {62 vars in 1318 bytes} [Fri Jun 10 13:07:50 2022] POST /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/action => generated 0 bytes in 231 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jun 10 13:07:50.724739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7a0c752a-51a5-4500-8275-852f8a00dc0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:50.725370 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7a0c752a-51a5-4500-8275-852f8a00dc0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:50.731433 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7a0c752a-51a5-4500-8275-852f8a00dc0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:50.731881 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7a0c752a-51a5-4500-8275-852f8a00dc0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:50.732646 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7a0c752a-51a5-4500-8275-852f8a00dc0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:50.733106 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7a0c752a-51a5-4500-8275-852f8a00dc0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:50.876533 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7a0c752a-51a5-4500-8275-852f8a00dc0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:50.885109 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7a0c752a-51a5-4500-8275-852f8a00dc0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:50.886593 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7a0c752a-51a5-4500-8275-852f8a00dc0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:50.887737 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7a0c752a-51a5-4500-8275-852f8a00dc0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1809 microversion: 2.20 time: 0.167029 Jun 10 13:07:50.888302 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 179/369] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:50 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1809 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:51.316880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3f192fc8-cc71-45a6-a4c9-00f3495f1c6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:51.321202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3f192fc8-cc71-45a6-a4c9-00f3495f1c6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:51.324509 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f192fc8-cc71-45a6-a4c9-00f3495f1c6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:51.325100 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f192fc8-cc71-45a6-a4c9-00f3495f1c6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:51.326609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f192fc8-cc71-45a6-a4c9-00f3495f1c6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:51.327293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f192fc8-cc71-45a6-a4c9-00f3495f1c6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:51.457650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f192fc8-cc71-45a6-a4c9-00f3495f1c6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:51.463558 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3f47356b-cfad-4c38-b64e-d9b8fedc161b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:51.464157 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3f47356b-cfad-4c38-b64e-d9b8fedc161b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:51.471548 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3f47356b-cfad-4c38-b64e-d9b8fedc161b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:51.471956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3f192fc8-cc71-45a6-a4c9-00f3495f1c6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.159950 Jun 10 13:07:51.472126 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3f47356b-cfad-4c38-b64e-d9b8fedc161b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:51.472532 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 191/370] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:51 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:51.473117 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3f47356b-cfad-4c38-b64e-d9b8fedc161b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:51.473558 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3f47356b-cfad-4c38-b64e-d9b8fedc161b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:51.535415 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3f47356b-cfad-4c38-b64e-d9b8fedc161b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:51.544758 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3f47356b-cfad-4c38-b64e-d9b8fedc161b tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1273 microversion: 2.1 time: 0.085047 Jun 10 13:07:51.545809 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 180/371] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:51 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1273 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:51.906897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-168a230e-6d3a-45d0-9978-0486381680ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:51.907602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-168a230e-6d3a-45d0-9978-0486381680ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:51.914567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-168a230e-6d3a-45d0-9978-0486381680ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:51.915012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-168a230e-6d3a-45d0-9978-0486381680ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:51.915784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-168a230e-6d3a-45d0-9978-0486381680ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:51.921036 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-168a230e-6d3a-45d0-9978-0486381680ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:52.055312 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-168a230e-6d3a-45d0-9978-0486381680ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:52.063723 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-168a230e-6d3a-45d0-9978-0486381680ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:52.065365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-168a230e-6d3a-45d0-9978-0486381680ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:52.066525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-168a230e-6d3a-45d0-9978-0486381680ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1809 microversion: 2.20 time: 0.164183 Jun 10 13:07:52.067072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 192/372] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:51 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1809 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:52.488949 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b512806d-f617-4a4d-8dce-030a001f2759 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:52.489552 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b512806d-f617-4a4d-8dce-030a001f2759 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:52.496008 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b512806d-f617-4a4d-8dce-030a001f2759 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:52.496473 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b512806d-f617-4a4d-8dce-030a001f2759 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:52.497245 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b512806d-f617-4a4d-8dce-030a001f2759 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:52.497670 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b512806d-f617-4a4d-8dce-030a001f2759 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:52.562019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bebd9fc6-4e9c-4544-95d0-cffa70c6e27c tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:52.562623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bebd9fc6-4e9c-4544-95d0-cffa70c6e27c tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:52.564988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:52.569835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bebd9fc6-4e9c-4544-95d0-cffa70c6e27c tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:52.570281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bebd9fc6-4e9c-4544-95d0-cffa70c6e27c tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:52.571044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bebd9fc6-4e9c-4544-95d0-cffa70c6e27c tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:52.571472 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bebd9fc6-4e9c-4544-95d0-cffa70c6e27c tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:52.628490 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b512806d-f617-4a4d-8dce-030a001f2759 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:52.634513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bebd9fc6-4e9c-4544-95d0-cffa70c6e27c tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:52.636160 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b512806d-f617-4a4d-8dce-030a001f2759 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.151198 Jun 10 13:07:52.636722 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 181/373] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:52 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:52.642568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bebd9fc6-4e9c-4544-95d0-cffa70c6e27c tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1273 microversion: 2.1 time: 0.084824 Jun 10 13:07:52.643127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 193/374] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:52 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1273 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:53.085034 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-60a77333-25d1-4d06-b569-8a37cf86b0da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:53.085648 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-60a77333-25d1-4d06-b569-8a37cf86b0da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:53.092663 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-60a77333-25d1-4d06-b569-8a37cf86b0da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:53.093998 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-60a77333-25d1-4d06-b569-8a37cf86b0da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:53.095330 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-60a77333-25d1-4d06-b569-8a37cf86b0da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:53.095863 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-60a77333-25d1-4d06-b569-8a37cf86b0da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:53.302099 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-60a77333-25d1-4d06-b569-8a37cf86b0da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:53.311226 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-60a77333-25d1-4d06-b569-8a37cf86b0da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:53.312840 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-60a77333-25d1-4d06-b569-8a37cf86b0da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:53.314075 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-60a77333-25d1-4d06-b569-8a37cf86b0da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1749 microversion: 2.20 time: 0.233510 Jun 10 13:07:53.314719 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 182/375] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:53 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1749 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:53.655651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7f715fd9-29fa-4376-b037-f20409145381 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:53.655651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7f715fd9-29fa-4376-b037-f20409145381 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:53.658745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=8 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:53.662537 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-aa866bd1-f27f-4549-a6dc-2468338a5a8d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:53.667211 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-aa866bd1-f27f-4549-a6dc-2468338a5a8d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:53.669272 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f715fd9-29fa-4376-b037-f20409145381 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:53.670398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f715fd9-29fa-4376-b037-f20409145381 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:53.672187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f715fd9-29fa-4376-b037-f20409145381 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:53.672859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f715fd9-29fa-4376-b037-f20409145381 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:53.683917 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-aa866bd1-f27f-4549-a6dc-2468338a5a8d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:53.684349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-aa866bd1-f27f-4549-a6dc-2468338a5a8d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:53.685881 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-aa866bd1-f27f-4549-a6dc-2468338a5a8d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:53.686325 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-aa866bd1-f27f-4549-a6dc-2468338a5a8d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:53.824467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7f715fd9-29fa-4376-b037-f20409145381 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:53.837126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7f715fd9-29fa-4376-b037-f20409145381 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.182879 Jun 10 13:07:53.837126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 194/376] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:53 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:53.846716 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-aa866bd1-f27f-4549-a6dc-2468338a5a8d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:53.854628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-aa866bd1-f27f-4549-a6dc-2468338a5a8d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.199732 Jun 10 13:07:53.855192 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 183/377] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:53 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:54.333807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-95f9d434-c395-4d59-87f8-70a9e7e6e480 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:54.334516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-95f9d434-c395-4d59-87f8-70a9e7e6e480 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:54.342642 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95f9d434-c395-4d59-87f8-70a9e7e6e480 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:54.343088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95f9d434-c395-4d59-87f8-70a9e7e6e480 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:54.344210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95f9d434-c395-4d59-87f8-70a9e7e6e480 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:54.348412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95f9d434-c395-4d59-87f8-70a9e7e6e480 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:54.475904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-95f9d434-c395-4d59-87f8-70a9e7e6e480 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:54.484364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-95f9d434-c395-4d59-87f8-70a9e7e6e480 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:54.486195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-95f9d434-c395-4d59-87f8-70a9e7e6e480 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:54.487337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-95f9d434-c395-4d59-87f8-70a9e7e6e480 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1751 microversion: 2.20 time: 0.158213 Jun 10 13:07:54.487895 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 195/378] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:54 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1751 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:54.650386 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-aabeeb4b-8e92-4482-a4b7-5f6dcdd7f44c req-f8faef6b-7938-402f-8732-763eb5e3a6ef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1d2dd021-a498-436e-8f04-13e3b612d24c", "tag": "e6ced7a0-efaf-4a5c-a6d4-e23d099b7ed8"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:07:54.658398 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-aabeeb4b-8e92-4482-a4b7-5f6dcdd7f44c req-f8faef6b-7938-402f-8732-763eb5e3a6ef service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:54.658830 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-aabeeb4b-8e92-4482-a4b7-5f6dcdd7f44c req-f8faef6b-7938-402f-8732-763eb5e3a6ef service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:54.669610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-aabeeb4b-8e92-4482-a4b7-5f6dcdd7f44c req-f8faef6b-7938-402f-8732-763eb5e3a6ef service nova] Creating event network-changed:e6ced7a0-efaf-4a5c-a6d4-e23d099b7ed8 for instance 1d2dd021-a498-436e-8f04-13e3b612d24c on hv-cinder-maste Jun 10 13:07:54.674417 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-aabeeb4b-8e92-4482-a4b7-5f6dcdd7f44c req-f8faef6b-7938-402f-8732-763eb5e3a6ef service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.027799 Jun 10 13:07:54.675095 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 184/379] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:07:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:07:54.851130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-be0f39df-62b2-42a9-8818-c471ca21f096 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:54.851659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-be0f39df-62b2-42a9-8818-c471ca21f096 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:54.858145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be0f39df-62b2-42a9-8818-c471ca21f096 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:54.858584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be0f39df-62b2-42a9-8818-c471ca21f096 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:54.859338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be0f39df-62b2-42a9-8818-c471ca21f096 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:54.859771 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be0f39df-62b2-42a9-8818-c471ca21f096 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:54.881959 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-abc4ed8e-893f-4b32-bf39-32b6446579ee tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:54.882653 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-abc4ed8e-893f-4b32-bf39-32b6446579ee tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:54.889478 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-abc4ed8e-893f-4b32-bf39-32b6446579ee tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:54.889920 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-abc4ed8e-893f-4b32-bf39-32b6446579ee tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:54.890680 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-abc4ed8e-893f-4b32-bf39-32b6446579ee tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:54.891108 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-abc4ed8e-893f-4b32-bf39-32b6446579ee tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:54.993383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-be0f39df-62b2-42a9-8818-c471ca21f096 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:55.001810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-be0f39df-62b2-42a9-8818-c471ca21f096 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.154432 Jun 10 13:07:55.002992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 196/380] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:54 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:55.031347 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-abc4ed8e-893f-4b32-bf39-32b6446579ee tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:55.038698 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-abc4ed8e-893f-4b32-bf39-32b6446579ee tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.161441 Jun 10 13:07:55.039252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 185/381] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:54 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:55.505642 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-918496ad-d042-4de4-a9df-c43c4c9f4a81 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:55.506249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-918496ad-d042-4de4-a9df-c43c4c9f4a81 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:55.512213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-918496ad-d042-4de4-a9df-c43c4c9f4a81 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:55.512678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-918496ad-d042-4de4-a9df-c43c4c9f4a81 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:55.514082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-918496ad-d042-4de4-a9df-c43c4c9f4a81 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:55.514520 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-918496ad-d042-4de4-a9df-c43c4c9f4a81 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:55.640416 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-918496ad-d042-4de4-a9df-c43c4c9f4a81 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:55.648674 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-918496ad-d042-4de4-a9df-c43c4c9f4a81 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:55.650185 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-918496ad-d042-4de4-a9df-c43c4c9f4a81 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:55.651326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-918496ad-d042-4de4-a9df-c43c4c9f4a81 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1751 microversion: 2.20 time: 0.149631 Jun 10 13:07:55.651914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 197/382] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:55 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1751 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:56.019554 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-83acb3f0-82df-4388-9f17-19ec35f13678 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:56.020161 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-83acb3f0-82df-4388-9f17-19ec35f13678 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:56.026621 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-83acb3f0-82df-4388-9f17-19ec35f13678 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:56.027065 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-83acb3f0-82df-4388-9f17-19ec35f13678 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:56.027822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-83acb3f0-82df-4388-9f17-19ec35f13678 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:56.028254 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-83acb3f0-82df-4388-9f17-19ec35f13678 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:56.055246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c61839d4-8c1f-46bd-9f49-b2d01f3b0997 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:56.055878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c61839d4-8c1f-46bd-9f49-b2d01f3b0997 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:56.062487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c61839d4-8c1f-46bd-9f49-b2d01f3b0997 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:56.062936 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c61839d4-8c1f-46bd-9f49-b2d01f3b0997 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:56.063698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c61839d4-8c1f-46bd-9f49-b2d01f3b0997 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:56.064132 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c61839d4-8c1f-46bd-9f49-b2d01f3b0997 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:56.079584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:56.156562 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-83acb3f0-82df-4388-9f17-19ec35f13678 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:56.164908 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-83acb3f0-82df-4388-9f17-19ec35f13678 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.149038 Jun 10 13:07:56.165611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 186/383] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:56 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:56.192647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c61839d4-8c1f-46bd-9f49-b2d01f3b0997 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:56.200512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c61839d4-8c1f-46bd-9f49-b2d01f3b0997 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.149021 Jun 10 13:07:56.201749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 198/384] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:56 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:56.670158 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-78d6416b-c82b-427d-b9fa-13cbaaf6ffab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:56.670754 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-78d6416b-c82b-427d-b9fa-13cbaaf6ffab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:56.676794 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-78d6416b-c82b-427d-b9fa-13cbaaf6ffab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:56.677247 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-78d6416b-c82b-427d-b9fa-13cbaaf6ffab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:56.677996 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-78d6416b-c82b-427d-b9fa-13cbaaf6ffab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:56.678419 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-78d6416b-c82b-427d-b9fa-13cbaaf6ffab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:56.811238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-78d6416b-c82b-427d-b9fa-13cbaaf6ffab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:56.820418 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-78d6416b-c82b-427d-b9fa-13cbaaf6ffab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:56.821930 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-78d6416b-c82b-427d-b9fa-13cbaaf6ffab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:56.823329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-78d6416b-c82b-427d-b9fa-13cbaaf6ffab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1751 microversion: 2.20 time: 0.157819 Jun 10 13:07:56.823885 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 187/385] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:56 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1751 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:57.181580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bd94010e-5487-40d9-bf56-46531a0f4228 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:57.182189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bd94010e-5487-40d9-bf56-46531a0f4228 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:57.189703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bd94010e-5487-40d9-bf56-46531a0f4228 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:57.190155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bd94010e-5487-40d9-bf56-46531a0f4228 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:57.190916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bd94010e-5487-40d9-bf56-46531a0f4228 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:57.191361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bd94010e-5487-40d9-bf56-46531a0f4228 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:57.217927 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1ba3356c-6ea1-47e4-ade8-1ee0f861bb57 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:57.218463 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1ba3356c-6ea1-47e4-ade8-1ee0f861bb57 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:57.225139 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1ba3356c-6ea1-47e4-ade8-1ee0f861bb57 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:57.225588 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1ba3356c-6ea1-47e4-ade8-1ee0f861bb57 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:57.226350 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1ba3356c-6ea1-47e4-ade8-1ee0f861bb57 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:57.226784 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1ba3356c-6ea1-47e4-ade8-1ee0f861bb57 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:57.364606 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1ba3356c-6ea1-47e4-ade8-1ee0f861bb57 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:57.372129 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1ba3356c-6ea1-47e4-ade8-1ee0f861bb57 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.158082 Jun 10 13:07:57.372700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 188/386] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:57 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:57.455499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bd94010e-5487-40d9-bf56-46531a0f4228 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:57.462832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bd94010e-5487-40d9-bf56-46531a0f4228 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.285176 Jun 10 13:07:57.463395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 199/387] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:57 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:57.841335 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-48490417-9c44-44df-b09e-e6e0f99105b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:57.842723 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-48490417-9c44-44df-b09e-e6e0f99105b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:57.848943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-48490417-9c44-44df-b09e-e6e0f99105b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:57.855739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-48490417-9c44-44df-b09e-e6e0f99105b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:57.855739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-48490417-9c44-44df-b09e-e6e0f99105b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:57.855739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-48490417-9c44-44df-b09e-e6e0f99105b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:58.070492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-48490417-9c44-44df-b09e-e6e0f99105b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:58.079479 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-48490417-9c44-44df-b09e-e6e0f99105b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:58.081011 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-48490417-9c44-44df-b09e-e6e0f99105b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:58.082166 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-48490417-9c44-44df-b09e-e6e0f99105b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1751 microversion: 2.20 time: 0.245063 Jun 10 13:07:58.082725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 189/388] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:57 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1751 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:58.390265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-170fc99d-1abb-4a58-a624-461a6f05569d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:58.390875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-170fc99d-1abb-4a58-a624-461a6f05569d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:58.397293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-170fc99d-1abb-4a58-a624-461a6f05569d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:58.397738 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-170fc99d-1abb-4a58-a624-461a6f05569d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:58.399825 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-170fc99d-1abb-4a58-a624-461a6f05569d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:58.400259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-170fc99d-1abb-4a58-a624-461a6f05569d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:58.479594 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6c4e37e8-e74e-4547-95eb-b555a04ef487 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:58.480930 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6c4e37e8-e74e-4547-95eb-b555a04ef487 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:58.487816 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c4e37e8-e74e-4547-95eb-b555a04ef487 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:58.488422 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c4e37e8-e74e-4547-95eb-b555a04ef487 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:58.489242 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c4e37e8-e74e-4547-95eb-b555a04ef487 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:58.489672 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c4e37e8-e74e-4547-95eb-b555a04ef487 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:58.532699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-170fc99d-1abb-4a58-a624-461a6f05569d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:58.542093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-170fc99d-1abb-4a58-a624-461a6f05569d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.156221 Jun 10 13:07:58.551192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 200/389] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:58 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:58.642111 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6c4e37e8-e74e-4547-95eb-b555a04ef487 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:58.649929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6c4e37e8-e74e-4547-95eb-b555a04ef487 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.174360 Jun 10 13:07:58.650593 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 190/390] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:58 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:07:59.100934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b8ebe0c6-b781-44c5-9fb1-67adf9585c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:59.101559 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b8ebe0c6-b781-44c5-9fb1-67adf9585c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:59.108104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8ebe0c6-b781-44c5-9fb1-67adf9585c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:59.108541 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8ebe0c6-b781-44c5-9fb1-67adf9585c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:59.109310 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8ebe0c6-b781-44c5-9fb1-67adf9585c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:59.109739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8ebe0c6-b781-44c5-9fb1-67adf9585c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:59.238059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b8ebe0c6-b781-44c5-9fb1-67adf9585c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:59.246327 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b8ebe0c6-b781-44c5-9fb1-67adf9585c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:59.247836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b8ebe0c6-b781-44c5-9fb1-67adf9585c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:59.249021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b8ebe0c6-b781-44c5-9fb1-67adf9585c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1751 microversion: 2.20 time: 0.152007 Jun 10 13:07:59.249578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 201/391] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:07:59 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1751 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:07:59.561141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1bea8445-93da-45f0-8a89-1907315c8ae9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:59.561744 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1bea8445-93da-45f0-8a89-1907315c8ae9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:59.568170 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1bea8445-93da-45f0-8a89-1907315c8ae9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:59.568972 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1bea8445-93da-45f0-8a89-1907315c8ae9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:59.569719 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1bea8445-93da-45f0-8a89-1907315c8ae9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:59.570142 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1bea8445-93da-45f0-8a89-1907315c8ae9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:59.666855 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-27a47d13-614b-4cd6-8c8e-a8ba5cbcbe0f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:07:59.667470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-27a47d13-614b-4cd6-8c8e-a8ba5cbcbe0f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:07:59.669867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:59.674703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-27a47d13-614b-4cd6-8c8e-a8ba5cbcbe0f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:59.675160 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-27a47d13-614b-4cd6-8c8e-a8ba5cbcbe0f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:59.675935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-27a47d13-614b-4cd6-8c8e-a8ba5cbcbe0f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:07:59.676373 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-27a47d13-614b-4cd6-8c8e-a8ba5cbcbe0f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:07:59.709538 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1bea8445-93da-45f0-8a89-1907315c8ae9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:59.717949 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1bea8445-93da-45f0-8a89-1907315c8ae9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.160737 Jun 10 13:07:59.718522 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 191/392] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:59 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:07:59.803848 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-27a47d13-614b-4cd6-8c8e-a8ba5cbcbe0f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:07:59.811505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-27a47d13-614b-4cd6-8c8e-a8ba5cbcbe0f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.148428 Jun 10 13:07:59.812079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 202/393] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:07:59 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:00.267136 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-37a1aa1d-eff8-450d-9efe-3cc5e0721fea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:00.267775 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-37a1aa1d-eff8-450d-9efe-3cc5e0721fea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:00.273861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37a1aa1d-eff8-450d-9efe-3cc5e0721fea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:00.274312 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37a1aa1d-eff8-450d-9efe-3cc5e0721fea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:00.275074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37a1aa1d-eff8-450d-9efe-3cc5e0721fea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:00.275513 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37a1aa1d-eff8-450d-9efe-3cc5e0721fea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:00.413854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-37a1aa1d-eff8-450d-9efe-3cc5e0721fea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:00.422550 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-37a1aa1d-eff8-450d-9efe-3cc5e0721fea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:00.424041 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-37a1aa1d-eff8-450d-9efe-3cc5e0721fea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:00.425409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-37a1aa1d-eff8-450d-9efe-3cc5e0721fea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1751 microversion: 2.20 time: 0.162229 Jun 10 13:08:00.425997 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 192/394] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:00 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1751 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:08:00.466314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1859816191", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-943261541", "security_groups": [{"name": "tempest-secgroup-smoke-1400256176"}], "networks": [{"uuid": "b28881e6-7982-476f-bf66-129fdb4fcbd7"}], "block_device_mapping_v2": [{"uuid": "19709d8b-58ed-4703-9e80-23e230cda52a", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:00.586903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.network.neutron [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] validate_networks() for [('b28881e6-7982-476f-bf66-129fdb4fcbd7', None, None, None, None, None)] {{(pid=118092) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:08:00.735748 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6c0b68e6-7081-4ea6-97a0-487d6ca75a25 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:00.736749 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6c0b68e6-7081-4ea6-97a0-487d6ca75a25 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:00.743372 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c0b68e6-7081-4ea6-97a0-487d6ca75a25 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:00.743826 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c0b68e6-7081-4ea6-97a0-487d6ca75a25 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:00.744588 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c0b68e6-7081-4ea6-97a0-487d6ca75a25 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:00.745056 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c0b68e6-7081-4ea6-97a0-487d6ca75a25 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:00.887936 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6c0b68e6-7081-4ea6-97a0-487d6ca75a25 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:00.895524 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6c0b68e6-7081-4ea6-97a0-487d6ca75a25 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.164374 Jun 10 13:08:00.896092 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 193/395] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:00 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:00.900648 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-baf08460-1c2b-45e6-bbb1-2cec65178b79 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:00.901284 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-baf08460-1c2b-45e6-bbb1-2cec65178b79 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:00.907798 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-baf08460-1c2b-45e6-bbb1-2cec65178b79 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:00.908252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-baf08460-1c2b-45e6-bbb1-2cec65178b79 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:00.909041 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-baf08460-1c2b-45e6-bbb1-2cec65178b79 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:00.909473 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-baf08460-1c2b-45e6-bbb1-2cec65178b79 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:00.913600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:00.980206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Flavor limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:08:00.980696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Image limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:08:00.981127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Flavor pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:08:00.981563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Image pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:08:00.981974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:08:00.991228 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:00.991679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:00.992478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:00.992949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:01.010050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Getting quotas for project 75273f09637240f3a5cc7129927765f1. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:08:01.013846 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Getting quotas for user 0a1368d42efc4543a908a61c48ec4472 and project 75273f09637240f3a5cc7129927765f1. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:08:01.039524 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-baf08460-1c2b-45e6-bbb1-2cec65178b79 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:01.047187 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-baf08460-1c2b-45e6-bbb1-2cec65178b79 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.150150 Jun 10 13:08:01.047744 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 194/396] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:00 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:01.071200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Going to run 1 instances... {{(pid=118092) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:08:01.073225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='19709d8b-58ed-4703-9e80-23e230cda52a',volume_size=None,volume_type=None)] {{(pid=118092) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:08:01.432929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-280c528d-b95f-4584-a1dc-126fa7f8fe8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.970673 Jun 10 13:08:01.439180 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 203/397] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:08:00 2022] POST /compute/v2.1/servers => generated 406 bytes in 972 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jun 10 13:08:01.444028 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-59d9ebaf-90aa-40fe-b18b-7f8e3593be3d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:01.444857 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-59d9ebaf-90aa-40fe-b18b-7f8e3593be3d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:01.451544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59d9ebaf-90aa-40fe-b18b-7f8e3593be3d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:01.452591 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59d9ebaf-90aa-40fe-b18b-7f8e3593be3d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:01.453809 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59d9ebaf-90aa-40fe-b18b-7f8e3593be3d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:01.454092 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59d9ebaf-90aa-40fe-b18b-7f8e3593be3d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:01.457724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-30fdec94-d5ed-4b9b-9c37-bbbdf90e2613 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:01.461243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-30fdec94-d5ed-4b9b-9c37-bbbdf90e2613 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:01.516285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-30fdec94-d5ed-4b9b-9c37-bbbdf90e2613 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:01.523418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-30fdec94-d5ed-4b9b-9c37-bbbdf90e2613 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1088 microversion: 2.1 time: 0.081703 Jun 10 13:08:01.524291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 204/398] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:01 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1088 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:01.672787 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-59d9ebaf-90aa-40fe-b18b-7f8e3593be3d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:01.682037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-59d9ebaf-90aa-40fe-b18b-7f8e3593be3d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:01.683642 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-59d9ebaf-90aa-40fe-b18b-7f8e3593be3d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:01.685047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-59d9ebaf-90aa-40fe-b18b-7f8e3593be3d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.244939 Jun 10 13:08:01.685679 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 195/399] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:01 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:08:01.911513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8ebff2c4-7b4a-4999-bb92-d07eb6e5add9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:01.912113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8ebff2c4-7b4a-4999-bb92-d07eb6e5add9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:01.918328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ebff2c4-7b4a-4999-bb92-d07eb6e5add9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:01.918858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ebff2c4-7b4a-4999-bb92-d07eb6e5add9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:01.919618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ebff2c4-7b4a-4999-bb92-d07eb6e5add9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:01.920049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ebff2c4-7b4a-4999-bb92-d07eb6e5add9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:02.059234 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8ebff2c4-7b4a-4999-bb92-d07eb6e5add9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:02.064155 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4b15a11f-890a-434c-8811-d400d4d32a8c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:02.064777 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4b15a11f-890a-434c-8811-d400d4d32a8c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:02.067717 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8ebff2c4-7b4a-4999-bb92-d07eb6e5add9 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.160022 Jun 10 13:08:02.068299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 205/400] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:01 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:02.071173 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b15a11f-890a-434c-8811-d400d4d32a8c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:02.071619 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b15a11f-890a-434c-8811-d400d4d32a8c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:02.072417 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b15a11f-890a-434c-8811-d400d4d32a8c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:02.072874 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b15a11f-890a-434c-8811-d400d4d32a8c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:02.207707 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4b15a11f-890a-434c-8811-d400d4d32a8c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:02.219330 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4b15a11f-890a-434c-8811-d400d4d32a8c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.159752 Jun 10 13:08:02.220433 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 196/401] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:02 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:02.539523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3e2e9e0d-fa36-48ed-b89a-7cb4e9a66c94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:02.540410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3e2e9e0d-fa36-48ed-b89a-7cb4e9a66c94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:02.547231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3e2e9e0d-fa36-48ed-b89a-7cb4e9a66c94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:02.547669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3e2e9e0d-fa36-48ed-b89a-7cb4e9a66c94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:02.548609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3e2e9e0d-fa36-48ed-b89a-7cb4e9a66c94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:02.549074 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3e2e9e0d-fa36-48ed-b89a-7cb4e9a66c94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:02.607740 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-72afcee0-3bdf-4030-a0b8-0c674da26171 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:02.608536 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-72afcee0-3bdf-4030-a0b8-0c674da26171 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:02.623322 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-72afcee0-3bdf-4030-a0b8-0c674da26171 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:02.623858 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-72afcee0-3bdf-4030-a0b8-0c674da26171 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:02.624810 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-72afcee0-3bdf-4030-a0b8-0c674da26171 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:02.625413 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-72afcee0-3bdf-4030-a0b8-0c674da26171 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:02.667226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3e2e9e0d-fa36-48ed-b89a-7cb4e9a66c94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:02.675628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3e2e9e0d-fa36-48ed-b89a-7cb4e9a66c94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1138 microversion: 2.1 time: 0.139957 Jun 10 13:08:02.686970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 206/402] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:02 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1138 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:02.704161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ee555ddb-3ac5-4309-b534-082e6e8c248b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:02.705158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ee555ddb-3ac5-4309-b534-082e6e8c248b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:02.712860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee555ddb-3ac5-4309-b534-082e6e8c248b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:02.713688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee555ddb-3ac5-4309-b534-082e6e8c248b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:02.718745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee555ddb-3ac5-4309-b534-082e6e8c248b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:02.718745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee555ddb-3ac5-4309-b534-082e6e8c248b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:02.789738 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-72afcee0-3bdf-4030-a0b8-0c674da26171 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lazy-loading 'services' on Instance uuid aeeb9a9c-292c-468a-8d2b-ef677e319ea7 {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:08:02.931605 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ee555ddb-3ac5-4309-b534-082e6e8c248b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:02.943193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ee555ddb-3ac5-4309-b534-082e6e8c248b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:02.944955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ee555ddb-3ac5-4309-b534-082e6e8c248b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:02.946749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ee555ddb-3ac5-4309-b534-082e6e8c248b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.247121 Jun 10 13:08:02.947304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 207/403] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:02 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:03.084905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b714400a-7b3c-41ea-8dd9-a7a98009b1ae tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:03.086298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b714400a-7b3c-41ea-8dd9-a7a98009b1ae tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:03.093996 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b714400a-7b3c-41ea-8dd9-a7a98009b1ae tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:03.094542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b714400a-7b3c-41ea-8dd9-a7a98009b1ae tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:03.095377 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b714400a-7b3c-41ea-8dd9-a7a98009b1ae tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:03.095971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b714400a-7b3c-41ea-8dd9-a7a98009b1ae tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:03.110581 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-72afcee0-3bdf-4030-a0b8-0c674da26171 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "DELETE /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7/os-volume_attachments/b7e78895-3e22-4baa-8b73-da6fa1f7d95b" status: 202 len: 0 microversion: 2.1 time: 0.507234 Jun 10 13:08:03.112486 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 197/404] 10.222.0.25 () {58 vars in 1415 bytes} [Fri Jun 10 13:08:02 2022] DELETE /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7/os-volume_attachments/b7e78895-3e22-4baa-8b73-da6fa1f7d95b => generated 0 bytes in 510 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jun 10 13:08:03.238690 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-08542f6e-5502-408c-80e1-2cd5ea220816 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:03.239788 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-08542f6e-5502-408c-80e1-2cd5ea220816 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:03.241931 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=99,nova_api:INSERT=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:03.253586 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-08542f6e-5502-408c-80e1-2cd5ea220816 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:03.258866 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-08542f6e-5502-408c-80e1-2cd5ea220816 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:03.260056 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-08542f6e-5502-408c-80e1-2cd5ea220816 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:03.260661 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-08542f6e-5502-408c-80e1-2cd5ea220816 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:03.266296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b714400a-7b3c-41ea-8dd9-a7a98009b1ae tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:03.274970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b714400a-7b3c-41ea-8dd9-a7a98009b1ae tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.194072 Jun 10 13:08:03.275863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 208/405] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:03 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:03.424321 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-08542f6e-5502-408c-80e1-2cd5ea220816 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:03.432221 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-08542f6e-5502-408c-80e1-2cd5ea220816 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.199095 Jun 10 13:08:03.432827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 198/406] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:03 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:03.694818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c0669ab1-1ab8-4902-a13d-6faad2dcc3ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:03.695433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c0669ab1-1ab8-4902-a13d-6faad2dcc3ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:03.701580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c0669ab1-1ab8-4902-a13d-6faad2dcc3ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:03.702032 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c0669ab1-1ab8-4902-a13d-6faad2dcc3ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:03.702916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c0669ab1-1ab8-4902-a13d-6faad2dcc3ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:03.703366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c0669ab1-1ab8-4902-a13d-6faad2dcc3ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:03.768623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c0669ab1-1ab8-4902-a13d-6faad2dcc3ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:03.776950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c0669ab1-1ab8-4902-a13d-6faad2dcc3ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1130 microversion: 2.1 time: 0.086354 Jun 10 13:08:03.777756 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 209/407] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:03 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1130 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:03.966676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-289e31c9-bc99-4176-85d4-22b0b56f47c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:03.968397 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-289e31c9-bc99-4176-85d4-22b0b56f47c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:03.979883 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-289e31c9-bc99-4176-85d4-22b0b56f47c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:03.980423 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-289e31c9-bc99-4176-85d4-22b0b56f47c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:03.981379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-7d40a103-bc23-42a4-ae0c-c2da44bd575b req-78e20aff-1c1a-4b12-95b1-146b98969ca6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1d2dd021-a498-436e-8f04-13e3b612d24c", "name": "network-vif-plugged", "status": "completed", "tag": "e6ced7a0-efaf-4a5c-a6d4-e23d099b7ed8"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:03.990606 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-289e31c9-bc99-4176-85d4-22b0b56f47c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:03.990606 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-289e31c9-bc99-4176-85d4-22b0b56f47c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:03.990979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-7d40a103-bc23-42a4-ae0c-c2da44bd575b req-78e20aff-1c1a-4b12-95b1-146b98969ca6 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:03.990979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-7d40a103-bc23-42a4-ae0c-c2da44bd575b req-78e20aff-1c1a-4b12-95b1-146b98969ca6 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:04.002947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-7d40a103-bc23-42a4-ae0c-c2da44bd575b req-78e20aff-1c1a-4b12-95b1-146b98969ca6 service nova] Creating event network-vif-plugged:e6ced7a0-efaf-4a5c-a6d4-e23d099b7ed8 for instance 1d2dd021-a498-436e-8f04-13e3b612d24c on hv-cinder-maste Jun 10 13:08:04.015418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-7d40a103-bc23-42a4-ae0c-c2da44bd575b req-78e20aff-1c1a-4b12-95b1-146b98969ca6 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036986 Jun 10 13:08:04.017516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 210/408] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:08:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:08:04.132456 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-289e31c9-bc99-4176-85d4-22b0b56f47c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:04.145638 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-289e31c9-bc99-4176-85d4-22b0b56f47c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:04.147150 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-289e31c9-bc99-4176-85d4-22b0b56f47c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:04.148447 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-289e31c9-bc99-4176-85d4-22b0b56f47c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.186489 Jun 10 13:08:04.149196 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 199/409] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:03 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:08:04.292940 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-08a067f5-d615-4e6a-9a6a-328d0750eaf1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:04.293973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-08a067f5-d615-4e6a-9a6a-328d0750eaf1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:04.302215 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-08a067f5-d615-4e6a-9a6a-328d0750eaf1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:04.302665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-08a067f5-d615-4e6a-9a6a-328d0750eaf1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:04.303445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-08a067f5-d615-4e6a-9a6a-328d0750eaf1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:04.303866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-08a067f5-d615-4e6a-9a6a-328d0750eaf1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:04.438217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-08a067f5-d615-4e6a-9a6a-328d0750eaf1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:04.446623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-08a067f5-d615-4e6a-9a6a-328d0750eaf1 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.158565 Jun 10 13:08:04.447184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 211/410] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:04 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:04.454931 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7fc20bfe-4250-45df-bd29-212db98183c3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:04.455542 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7fc20bfe-4250-45df-bd29-212db98183c3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:04.461598 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7fc20bfe-4250-45df-bd29-212db98183c3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:04.462049 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7fc20bfe-4250-45df-bd29-212db98183c3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:04.462812 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7fc20bfe-4250-45df-bd29-212db98183c3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:04.463246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7fc20bfe-4250-45df-bd29-212db98183c3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:04.603787 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7fc20bfe-4250-45df-bd29-212db98183c3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:04.610717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7fc20bfe-4250-45df-bd29-212db98183c3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.163422 Jun 10 13:08:04.611551 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 200/411] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:04 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:04.794428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1c5ac415-0d9e-45c0-9288-7533feee60b1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:04.795038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1c5ac415-0d9e-45c0-9288-7533feee60b1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:04.801188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c5ac415-0d9e-45c0-9288-7533feee60b1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:04.801577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c5ac415-0d9e-45c0-9288-7533feee60b1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:04.802337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c5ac415-0d9e-45c0-9288-7533feee60b1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:04.802773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c5ac415-0d9e-45c0-9288-7533feee60b1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:04.866393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1c5ac415-0d9e-45c0-9288-7533feee60b1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:04.874106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1c5ac415-0d9e-45c0-9288-7533feee60b1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1130 microversion: 2.1 time: 0.083508 Jun 10 13:08:04.874680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 212/412] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:04 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1130 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:05.167465 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4d73a659-bdf4-481b-83b2-2e7b3deeafce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:05.168004 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4d73a659-bdf4-481b-83b2-2e7b3deeafce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:05.174779 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4d73a659-bdf4-481b-83b2-2e7b3deeafce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:05.175227 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4d73a659-bdf4-481b-83b2-2e7b3deeafce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:05.175996 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4d73a659-bdf4-481b-83b2-2e7b3deeafce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:05.176426 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4d73a659-bdf4-481b-83b2-2e7b3deeafce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:05.333071 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4d73a659-bdf4-481b-83b2-2e7b3deeafce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:05.349933 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4d73a659-bdf4-481b-83b2-2e7b3deeafce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:05.351722 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4d73a659-bdf4-481b-83b2-2e7b3deeafce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:05.352908 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4d73a659-bdf4-481b-83b2-2e7b3deeafce tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.189414 Jun 10 13:08:05.353610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 201/413] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:05 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:08:05.469433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f94e2ac2-61f7-4120-ae60-750add85595d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:05.470046 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f94e2ac2-61f7-4120-ae60-750add85595d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:05.476251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f94e2ac2-61f7-4120-ae60-750add85595d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:05.476696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f94e2ac2-61f7-4120-ae60-750add85595d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:05.477495 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f94e2ac2-61f7-4120-ae60-750add85595d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:05.477927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f94e2ac2-61f7-4120-ae60-750add85595d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:05.616883 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f94e2ac2-61f7-4120-ae60-750add85595d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:05.627120 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f94e2ac2-61f7-4120-ae60-750add85595d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.166284 Jun 10 13:08:05.627872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 213/414] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:05 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:05.628759 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7732eac2-0762-4d7d-b163-f68e0eb20ce5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:05.629475 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7732eac2-0762-4d7d-b163-f68e0eb20ce5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:05.635815 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7732eac2-0762-4d7d-b163-f68e0eb20ce5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:05.636361 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7732eac2-0762-4d7d-b163-f68e0eb20ce5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:05.637241 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7732eac2-0762-4d7d-b163-f68e0eb20ce5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:05.637786 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7732eac2-0762-4d7d-b163-f68e0eb20ce5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:05.770886 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7732eac2-0762-4d7d-b163-f68e0eb20ce5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:05.778851 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7732eac2-0762-4d7d-b163-f68e0eb20ce5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.155178 Jun 10 13:08:05.779419 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 202/415] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:05 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:05.890798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-209b78e5-5af0-463a-b252-7f7f3184a4e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:05.891769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-209b78e5-5af0-463a-b252-7f7f3184a4e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:05.899108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-209b78e5-5af0-463a-b252-7f7f3184a4e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:05.899811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-209b78e5-5af0-463a-b252-7f7f3184a4e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:05.900882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-209b78e5-5af0-463a-b252-7f7f3184a4e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:05.901441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-209b78e5-5af0-463a-b252-7f7f3184a4e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:05.969096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-209b78e5-5af0-463a-b252-7f7f3184a4e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:05.976677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-209b78e5-5af0-463a-b252-7f7f3184a4e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1130 microversion: 2.1 time: 0.090205 Jun 10 13:08:05.977491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 214/416] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:05 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1130 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:06.375192 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0913f8f8-41c8-40e8-bdcd-7243ad01bc68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:06.375816 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0913f8f8-41c8-40e8-bdcd-7243ad01bc68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:06.396795 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0913f8f8-41c8-40e8-bdcd-7243ad01bc68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:06.397263 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0913f8f8-41c8-40e8-bdcd-7243ad01bc68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:06.398046 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0913f8f8-41c8-40e8-bdcd-7243ad01bc68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:06.399289 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0913f8f8-41c8-40e8-bdcd-7243ad01bc68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:06.568983 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0913f8f8-41c8-40e8-bdcd-7243ad01bc68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:06.581652 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0913f8f8-41c8-40e8-bdcd-7243ad01bc68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:06.583243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0913f8f8-41c8-40e8-bdcd-7243ad01bc68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:06.584420 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0913f8f8-41c8-40e8-bdcd-7243ad01bc68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.216463 Jun 10 13:08:06.585204 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 203/417] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:06 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:06.645302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e14ce1af-ef7a-44f7-a094-773bfaa0049a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:06.645866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e14ce1af-ef7a-44f7-a094-773bfaa0049a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:06.652374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e14ce1af-ef7a-44f7-a094-773bfaa0049a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:06.652859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e14ce1af-ef7a-44f7-a094-773bfaa0049a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:06.653641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e14ce1af-ef7a-44f7-a094-773bfaa0049a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:06.654082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e14ce1af-ef7a-44f7-a094-773bfaa0049a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:06.786187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e14ce1af-ef7a-44f7-a094-773bfaa0049a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:06.793863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e14ce1af-ef7a-44f7-a094-773bfaa0049a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.152460 Jun 10 13:08:06.794594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 215/418] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:06 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:06.795646 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2599d746-2249-4b83-81ad-6e9723eeb9e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:06.796534 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2599d746-2249-4b83-81ad-6e9723eeb9e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:06.803347 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2599d746-2249-4b83-81ad-6e9723eeb9e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:06.803922 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2599d746-2249-4b83-81ad-6e9723eeb9e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:06.804819 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2599d746-2249-4b83-81ad-6e9723eeb9e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:06.805377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2599d746-2249-4b83-81ad-6e9723eeb9e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:06.941434 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2599d746-2249-4b83-81ad-6e9723eeb9e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:06.950056 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2599d746-2249-4b83-81ad-6e9723eeb9e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.158326 Jun 10 13:08:06.951295 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 204/419] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:06 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:06.994141 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3706a331-8207-489b-a043-33b80997e2db tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:06.994746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3706a331-8207-489b-a043-33b80997e2db tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:07.001576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3706a331-8207-489b-a043-33b80997e2db tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:07.011666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3706a331-8207-489b-a043-33b80997e2db tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:07.011666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3706a331-8207-489b-a043-33b80997e2db tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:07.011666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3706a331-8207-489b-a043-33b80997e2db tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:07.067575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3706a331-8207-489b-a043-33b80997e2db tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:07.079122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3706a331-8207-489b-a043-33b80997e2db tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1130 microversion: 2.1 time: 0.088485 Jun 10 13:08:07.079887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 216/420] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:06 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1130 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:07.603365 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a62397d2-34ca-4533-a140-4b3f66e80b7d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:07.604051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a62397d2-34ca-4533-a140-4b3f66e80b7d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:07.610710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a62397d2-34ca-4533-a140-4b3f66e80b7d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:07.611454 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a62397d2-34ca-4533-a140-4b3f66e80b7d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:07.612223 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a62397d2-34ca-4533-a140-4b3f66e80b7d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:07.612653 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a62397d2-34ca-4533-a140-4b3f66e80b7d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:07.814878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-acd727f5-15a2-47f8-a7a6-07a5fba83771 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:07.816050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-acd727f5-15a2-47f8-a7a6-07a5fba83771 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:07.823697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acd727f5-15a2-47f8-a7a6-07a5fba83771 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:07.824366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acd727f5-15a2-47f8-a7a6-07a5fba83771 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:07.825618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acd727f5-15a2-47f8-a7a6-07a5fba83771 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:07.826346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acd727f5-15a2-47f8-a7a6-07a5fba83771 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:07.851827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:07.993800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-acd727f5-15a2-47f8-a7a6-07a5fba83771 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:08.002487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-acd727f5-15a2-47f8-a7a6-07a5fba83771 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.195136 Jun 10 13:08:08.012058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 217/421] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:07 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:08.012353 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a62397d2-34ca-4533-a140-4b3f66e80b7d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:08.012480 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3bc02b0b-d5bb-4403-a222-8eddffcbd595 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:08.012480 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3bc02b0b-d5bb-4403-a222-8eddffcbd595 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:08.015645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3bc02b0b-d5bb-4403-a222-8eddffcbd595 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:08.015837 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a62397d2-34ca-4533-a140-4b3f66e80b7d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:08.016239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3bc02b0b-d5bb-4403-a222-8eddffcbd595 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:08.017115 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3bc02b0b-d5bb-4403-a222-8eddffcbd595 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:08.017489 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a62397d2-34ca-4533-a140-4b3f66e80b7d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:08.017621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3bc02b0b-d5bb-4403-a222-8eddffcbd595 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:08.019507 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a62397d2-34ca-4533-a140-4b3f66e80b7d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.419727 Jun 10 13:08:08.019507 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 205/422] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:07 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:08:08.099054 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7d831f88-7456-4e7c-841d-cd45a525f608 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:08.099831 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7d831f88-7456-4e7c-841d-cd45a525f608 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:08.107093 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7d831f88-7456-4e7c-841d-cd45a525f608 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:08.109827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7d831f88-7456-4e7c-841d-cd45a525f608 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:08.109827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7d831f88-7456-4e7c-841d-cd45a525f608 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:08.109827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7d831f88-7456-4e7c-841d-cd45a525f608 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:08.172080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3bc02b0b-d5bb-4403-a222-8eddffcbd595 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:08.177634 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7d831f88-7456-4e7c-841d-cd45a525f608 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:08.181941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3bc02b0b-d5bb-4403-a222-8eddffcbd595 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.177797 Jun 10 13:08:08.182728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 218/423] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:08 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:08.185904 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7d831f88-7456-4e7c-841d-cd45a525f608 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1130 microversion: 2.1 time: 0.090723 Jun 10 13:08:08.191757 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 206/424] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:08 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1130 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:09.020125 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-299a50cf-f90f-4a70-b8ab-f12e9030d316 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:09.020881 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-299a50cf-f90f-4a70-b8ab-f12e9030d316 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:09.028112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-299a50cf-f90f-4a70-b8ab-f12e9030d316 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:09.028562 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-299a50cf-f90f-4a70-b8ab-f12e9030d316 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:09.029346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-299a50cf-f90f-4a70-b8ab-f12e9030d316 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:09.029783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-299a50cf-f90f-4a70-b8ab-f12e9030d316 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:09.037507 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e5099cae-be15-4a34-89f8-a6f2f9c88c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:09.040291 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e5099cae-be15-4a34-89f8-a6f2f9c88c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:09.045363 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e5099cae-be15-4a34-89f8-a6f2f9c88c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:09.045820 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e5099cae-be15-4a34-89f8-a6f2f9c88c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:09.046589 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e5099cae-be15-4a34-89f8-a6f2f9c88c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:09.047026 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e5099cae-be15-4a34-89f8-a6f2f9c88c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:09.195347 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e5099cae-be15-4a34-89f8-a6f2f9c88c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:09.200527 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-299a50cf-f90f-4a70-b8ab-f12e9030d316 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:09.205494 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e5099cae-be15-4a34-89f8-a6f2f9c88c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:09.207133 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e5099cae-be15-4a34-89f8-a6f2f9c88c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:09.208681 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e5099cae-be15-4a34-89f8-a6f2f9c88c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.174958 Jun 10 13:08:09.209277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-299a50cf-f90f-4a70-b8ab-f12e9030d316 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.192934 Jun 10 13:08:09.209421 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 207/425] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:09 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:09.210150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 219/426] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:09 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:09.214756 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-386750d7-6770-40c3-9ef2-b295536cedb4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:09.215370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-386750d7-6770-40c3-9ef2-b295536cedb4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:09.221447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-386750d7-6770-40c3-9ef2-b295536cedb4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:09.222023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-386750d7-6770-40c3-9ef2-b295536cedb4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:09.222863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-386750d7-6770-40c3-9ef2-b295536cedb4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:09.223385 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-386750d7-6770-40c3-9ef2-b295536cedb4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:09.226948 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-afe663d0-386d-4d78-90b4-00c898edcac7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:09.228813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-afe663d0-386d-4d78-90b4-00c898edcac7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:09.234713 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-afe663d0-386d-4d78-90b4-00c898edcac7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:09.235173 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-afe663d0-386d-4d78-90b4-00c898edcac7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:09.235946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-afe663d0-386d-4d78-90b4-00c898edcac7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:09.236632 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-afe663d0-386d-4d78-90b4-00c898edcac7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:09.294814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-386750d7-6770-40c3-9ef2-b295536cedb4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:09.305832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-386750d7-6770-40c3-9ef2-b295536cedb4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1130 microversion: 2.1 time: 0.094391 Jun 10 13:08:09.306684 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 220/427] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:09 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1130 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:09.387024 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-afe663d0-386d-4d78-90b4-00c898edcac7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:09.394992 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-afe663d0-386d-4d78-90b4-00c898edcac7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.171827 Jun 10 13:08:09.395556 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 208/428] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:09 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:10.228170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7426d50a-87d0-44dd-8ffe-cce72737ea8a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:10.228824 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a28958a8-5b7d-4b46-9bca-72a1fdbbc0a9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:10.229216 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7426d50a-87d0-44dd-8ffe-cce72737ea8a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:10.229645 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a28958a8-5b7d-4b46-9bca-72a1fdbbc0a9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:10.235415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7426d50a-87d0-44dd-8ffe-cce72737ea8a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:10.235860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7426d50a-87d0-44dd-8ffe-cce72737ea8a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:10.237012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7426d50a-87d0-44dd-8ffe-cce72737ea8a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:10.237619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7426d50a-87d0-44dd-8ffe-cce72737ea8a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:10.237782 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a28958a8-5b7d-4b46-9bca-72a1fdbbc0a9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:10.238554 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a28958a8-5b7d-4b46-9bca-72a1fdbbc0a9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:10.240049 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a28958a8-5b7d-4b46-9bca-72a1fdbbc0a9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:10.240592 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a28958a8-5b7d-4b46-9bca-72a1fdbbc0a9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:10.458130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7426d50a-87d0-44dd-8ffe-cce72737ea8a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:10.458130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7426d50a-87d0-44dd-8ffe-cce72737ea8a tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.196001 Jun 10 13:08:10.458130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 221/429] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:10 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:10.458130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2c00baf9-5d36-4747-99de-86e0804ff64d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:10.458506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2c00baf9-5d36-4747-99de-86e0804ff64d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:10.458506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2c00baf9-5d36-4747-99de-86e0804ff64d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:10.458506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2c00baf9-5d36-4747-99de-86e0804ff64d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:10.458727 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a28958a8-5b7d-4b46-9bca-72a1fdbbc0a9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:10.458836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2c00baf9-5d36-4747-99de-86e0804ff64d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:10.458836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2c00baf9-5d36-4747-99de-86e0804ff64d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:10.458983 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a28958a8-5b7d-4b46-9bca-72a1fdbbc0a9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:10.458983 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a28958a8-5b7d-4b46-9bca-72a1fdbbc0a9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:10.458983 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a28958a8-5b7d-4b46-9bca-72a1fdbbc0a9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.220979 Jun 10 13:08:10.459219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 209/430] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:10 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:10.459219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-34903484-1b2b-4278-8ffb-2c03ebaed8bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:10.459219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-34903484-1b2b-4278-8ffb-2c03ebaed8bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:10.480879 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-34903484-1b2b-4278-8ffb-2c03ebaed8bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:10.481249 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-34903484-1b2b-4278-8ffb-2c03ebaed8bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:10.482061 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-34903484-1b2b-4278-8ffb-2c03ebaed8bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:10.482483 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-34903484-1b2b-4278-8ffb-2c03ebaed8bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:10.518638 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:10.523980 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2c00baf9-5d36-4747-99de-86e0804ff64d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:10.531649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2c00baf9-5d36-4747-99de-86e0804ff64d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1130 microversion: 2.1 time: 0.111243 Jun 10 13:08:10.542767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 222/431] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:10 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1130 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:10.644890 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-34903484-1b2b-4278-8ffb-2c03ebaed8bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:10.652481 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-34903484-1b2b-4278-8ffb-2c03ebaed8bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.205998 Jun 10 13:08:10.657315 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 210/432] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:10 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:11.480183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1b0d88ff-fa66-460e-8c95-c0b0cf07537d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:11.480818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1b0d88ff-fa66-460e-8c95-c0b0cf07537d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:11.483077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:11.490255 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b02605a5-d045-4a07-95a1-2b5706b646c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:11.491116 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b02605a5-d045-4a07-95a1-2b5706b646c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:11.495032 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b0d88ff-fa66-460e-8c95-c0b0cf07537d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:11.496111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b0d88ff-fa66-460e-8c95-c0b0cf07537d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:11.505398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b0d88ff-fa66-460e-8c95-c0b0cf07537d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:11.505928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b0d88ff-fa66-460e-8c95-c0b0cf07537d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:11.506360 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b02605a5-d045-4a07-95a1-2b5706b646c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:11.507065 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b02605a5-d045-4a07-95a1-2b5706b646c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:11.507900 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b02605a5-d045-4a07-95a1-2b5706b646c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:11.508345 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b02605a5-d045-4a07-95a1-2b5706b646c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:11.530441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=8 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:11.687674 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b02605a5-d045-4a07-95a1-2b5706b646c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:11.688113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1b0d88ff-fa66-460e-8c95-c0b0cf07537d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:11.697743 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b02605a5-d045-4a07-95a1-2b5706b646c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:11.697986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1b0d88ff-fa66-460e-8c95-c0b0cf07537d tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.221577 Jun 10 13:08:11.698761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 223/433] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:11 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:11.702117 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b02605a5-d045-4a07-95a1-2b5706b646c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:11.703878 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b02605a5-d045-4a07-95a1-2b5706b646c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.217727 Jun 10 13:08:11.712838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a55fe7eb-9582-4583-a83a-e3c4a95a3d21 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:11.713453 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a55fe7eb-9582-4583-a83a-e3c4a95a3d21 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:11.715657 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 211/434] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:11 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:11.720433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a55fe7eb-9582-4583-a83a-e3c4a95a3d21 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:11.720847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a55fe7eb-9582-4583-a83a-e3c4a95a3d21 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:11.721894 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a55fe7eb-9582-4583-a83a-e3c4a95a3d21 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:11.722332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a55fe7eb-9582-4583-a83a-e3c4a95a3d21 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:11.722893 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1ab792cd-f53f-4a7a-a790-72b458c8a458 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:11.723622 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1ab792cd-f53f-4a7a-a790-72b458c8a458 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:11.730097 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1ab792cd-f53f-4a7a-a790-72b458c8a458 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:11.730607 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1ab792cd-f53f-4a7a-a790-72b458c8a458 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:11.731451 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1ab792cd-f53f-4a7a-a790-72b458c8a458 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:11.731943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1ab792cd-f53f-4a7a-a790-72b458c8a458 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:11.795389 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a55fe7eb-9582-4583-a83a-e3c4a95a3d21 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:11.806642 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a55fe7eb-9582-4583-a83a-e3c4a95a3d21 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1130 microversion: 2.1 time: 0.097438 Jun 10 13:08:11.807408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 224/435] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:11 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1130 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:11.875686 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1ab792cd-f53f-4a7a-a790-72b458c8a458 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:11.884533 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1ab792cd-f53f-4a7a-a790-72b458c8a458 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.167132 Jun 10 13:08:11.886038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 212/436] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:11 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:12.715122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-80889ebb-7219-42cf-8bbf-8174a538e433 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:12.715725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-80889ebb-7219-42cf-8bbf-8174a538e433 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:12.722291 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2186bc4f-6b53-4450-b7af-08a4dcce6a72 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:12.722758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80889ebb-7219-42cf-8bbf-8174a538e433 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:12.722896 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2186bc4f-6b53-4450-b7af-08a4dcce6a72 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:12.723015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80889ebb-7219-42cf-8bbf-8174a538e433 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:12.724195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80889ebb-7219-42cf-8bbf-8174a538e433 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:12.724646 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80889ebb-7219-42cf-8bbf-8174a538e433 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:12.729517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2186bc4f-6b53-4450-b7af-08a4dcce6a72 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:12.729964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2186bc4f-6b53-4450-b7af-08a4dcce6a72 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:12.730718 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2186bc4f-6b53-4450-b7af-08a4dcce6a72 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:12.731150 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2186bc4f-6b53-4450-b7af-08a4dcce6a72 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:12.862984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-80889ebb-7219-42cf-8bbf-8174a538e433 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:12.864983 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2186bc4f-6b53-4450-b7af-08a4dcce6a72 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:12.870552 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-80889ebb-7219-42cf-8bbf-8174a538e433 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.159227 Jun 10 13:08:12.871123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 225/437] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:12 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:12.873817 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2186bc4f-6b53-4450-b7af-08a4dcce6a72 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:12.878574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2186bc4f-6b53-4450-b7af-08a4dcce6a72 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:12.878703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cebf9ce3-f13b-40a1-a170-6b71c08b5aed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:12.878703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cebf9ce3-f13b-40a1-a170-6b71c08b5aed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:12.878870 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2186bc4f-6b53-4450-b7af-08a4dcce6a72 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.158561 Jun 10 13:08:12.878870 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 213/438] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:12 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:12.882656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cebf9ce3-f13b-40a1-a170-6b71c08b5aed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:12.883102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cebf9ce3-f13b-40a1-a170-6b71c08b5aed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:12.883892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cebf9ce3-f13b-40a1-a170-6b71c08b5aed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:12.884329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cebf9ce3-f13b-40a1-a170-6b71c08b5aed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:12.901365 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-95b8dab6-f6e7-4afd-a77a-96e264fd34e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:12.901970 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-95b8dab6-f6e7-4afd-a77a-96e264fd34e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:12.908333 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-95b8dab6-f6e7-4afd-a77a-96e264fd34e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:12.908812 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-95b8dab6-f6e7-4afd-a77a-96e264fd34e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:12.909585 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-95b8dab6-f6e7-4afd-a77a-96e264fd34e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:12.910017 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-95b8dab6-f6e7-4afd-a77a-96e264fd34e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:12.951742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cebf9ce3-f13b-40a1-a170-6b71c08b5aed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:12.959551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cebf9ce3-f13b-40a1-a170-6b71c08b5aed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1130 microversion: 2.1 time: 0.087420 Jun 10 13:08:12.960128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 226/439] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:12 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1130 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:13.045614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-95b8dab6-f6e7-4afd-a77a-96e264fd34e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:13.054199 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-95b8dab6-f6e7-4afd-a77a-96e264fd34e6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.156657 Jun 10 13:08:13.055029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 214/440] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:12 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:13.887149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7edd7aa9-ab6b-4081-98f8-6341f4a43bfa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:13.887755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7edd7aa9-ab6b-4081-98f8-6341f4a43bfa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:13.894805 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7edd7aa9-ab6b-4081-98f8-6341f4a43bfa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:13.895249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7edd7aa9-ab6b-4081-98f8-6341f4a43bfa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:13.895508 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-67b02ec6-1fc2-4b68-8d10-7cf9552cfa3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:13.896037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7edd7aa9-ab6b-4081-98f8-6341f4a43bfa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:13.896140 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-67b02ec6-1fc2-4b68-8d10-7cf9552cfa3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:13.896560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7edd7aa9-ab6b-4081-98f8-6341f4a43bfa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:13.902779 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-67b02ec6-1fc2-4b68-8d10-7cf9552cfa3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:13.903256 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-67b02ec6-1fc2-4b68-8d10-7cf9552cfa3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:13.904017 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-67b02ec6-1fc2-4b68-8d10-7cf9552cfa3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:13.904449 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-67b02ec6-1fc2-4b68-8d10-7cf9552cfa3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:14.033655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7edd7aa9-ab6b-4081-98f8-6341f4a43bfa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:14.034993 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-67b02ec6-1fc2-4b68-8d10-7cf9552cfa3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:14.041100 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7edd7aa9-ab6b-4081-98f8-6341f4a43bfa tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.157643 Jun 10 13:08:14.041661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 227/441] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:13 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:14.043402 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-67b02ec6-1fc2-4b68-8d10-7cf9552cfa3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:14.044940 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-67b02ec6-1fc2-4b68-8d10-7cf9552cfa3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:14.046023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7f690dbe-820b-4566-82d0-8699f4740ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:14.046183 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-67b02ec6-1fc2-4b68-8d10-7cf9552cfa3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.154435 Jun 10 13:08:14.046808 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7f690dbe-820b-4566-82d0-8699f4740ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:14.046927 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 215/442] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:13 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:08:14.052809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f690dbe-820b-4566-82d0-8699f4740ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:14.053253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f690dbe-820b-4566-82d0-8699f4740ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:14.054007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f690dbe-820b-4566-82d0-8699f4740ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:14.054432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f690dbe-820b-4566-82d0-8699f4740ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:14.071193 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-967c3640-4465-4cf9-9f17-e63ef7de0f40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:14.071898 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-967c3640-4465-4cf9-9f17-e63ef7de0f40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:14.078619 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-967c3640-4465-4cf9-9f17-e63ef7de0f40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:14.079012 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-967c3640-4465-4cf9-9f17-e63ef7de0f40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:14.079775 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-967c3640-4465-4cf9-9f17-e63ef7de0f40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:14.080209 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-967c3640-4465-4cf9-9f17-e63ef7de0f40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:14.124916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7f690dbe-820b-4566-82d0-8699f4740ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:14.131858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7f690dbe-820b-4566-82d0-8699f4740ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1186 microversion: 2.1 time: 0.089228 Jun 10 13:08:14.132429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 228/443] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:14 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1186 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:14.214053 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-967c3640-4465-4cf9-9f17-e63ef7de0f40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:14.221424 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-967c3640-4465-4cf9-9f17-e63ef7de0f40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.153933 Jun 10 13:08:14.221982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 216/444] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:14 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:15.058183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8eccd0b7-26f0-451c-ab51-b5d25d409b52 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:15.058701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8eccd0b7-26f0-451c-ab51-b5d25d409b52 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:15.063437 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a552e552-c4b7-4ba2-971d-df192266a905 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:15.064097 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a552e552-c4b7-4ba2-971d-df192266a905 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:15.065469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8eccd0b7-26f0-451c-ab51-b5d25d409b52 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:15.065931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8eccd0b7-26f0-451c-ab51-b5d25d409b52 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:15.066696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8eccd0b7-26f0-451c-ab51-b5d25d409b52 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:15.067137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8eccd0b7-26f0-451c-ab51-b5d25d409b52 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:15.070484 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a552e552-c4b7-4ba2-971d-df192266a905 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:15.070937 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a552e552-c4b7-4ba2-971d-df192266a905 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:15.071693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a552e552-c4b7-4ba2-971d-df192266a905 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:15.072124 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a552e552-c4b7-4ba2-971d-df192266a905 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:15.196392 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8eccd0b7-26f0-451c-ab51-b5d25d409b52 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:15.198605 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a552e552-c4b7-4ba2-971d-df192266a905 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:15.204602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8eccd0b7-26f0-451c-ab51-b5d25d409b52 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.150354 Jun 10 13:08:15.205295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 229/445] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:15 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:15.207903 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a552e552-c4b7-4ba2-971d-df192266a905 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:15.209420 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a552e552-c4b7-4ba2-971d-df192266a905 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:15.210559 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a552e552-c4b7-4ba2-971d-df192266a905 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.150937 Jun 10 13:08:15.211092 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 217/446] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:15 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:15.215936 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6be1749d-e596-4450-8b28-f4bad6a75edf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:15.216543 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6be1749d-e596-4450-8b28-f4bad6a75edf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:15.223048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6be1749d-e596-4450-8b28-f4bad6a75edf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:15.223500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6be1749d-e596-4450-8b28-f4bad6a75edf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:15.224275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6be1749d-e596-4450-8b28-f4bad6a75edf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:15.224704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6be1749d-e596-4450-8b28-f4bad6a75edf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:15.238185 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3e29a7c0-8df1-4bdd-b9d8-18f8e29d160f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:15.238785 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3e29a7c0-8df1-4bdd-b9d8-18f8e29d160f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:15.244620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3e29a7c0-8df1-4bdd-b9d8-18f8e29d160f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:15.245095 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3e29a7c0-8df1-4bdd-b9d8-18f8e29d160f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:15.245844 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3e29a7c0-8df1-4bdd-b9d8-18f8e29d160f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:15.246268 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3e29a7c0-8df1-4bdd-b9d8-18f8e29d160f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:15.285326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6be1749d-e596-4450-8b28-f4bad6a75edf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:15.292744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6be1749d-e596-4450-8b28-f4bad6a75edf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1194 microversion: 2.1 time: 0.080427 Jun 10 13:08:15.293352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 230/447] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:15 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1194 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:15.375574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3e29a7c0-8df1-4bdd-b9d8-18f8e29d160f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:15.383039 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3e29a7c0-8df1-4bdd-b9d8-18f8e29d160f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.148767 Jun 10 13:08:15.383605 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 218/448] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:15 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:16.221973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6e270575-3a08-47e2-b09e-fa54d2c8a9e3 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:16.222725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6e270575-3a08-47e2-b09e-fa54d2c8a9e3 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:16.228939 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3d1c6b3a-1d26-42f2-bee6-9bac5c001788 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:16.229388 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e270575-3a08-47e2-b09e-fa54d2c8a9e3 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:16.229539 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3d1c6b3a-1d26-42f2-bee6-9bac5c001788 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:16.229767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e270575-3a08-47e2-b09e-fa54d2c8a9e3 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:16.230486 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e270575-3a08-47e2-b09e-fa54d2c8a9e3 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:16.230918 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e270575-3a08-47e2-b09e-fa54d2c8a9e3 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:16.236382 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d1c6b3a-1d26-42f2-bee6-9bac5c001788 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:16.236866 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d1c6b3a-1d26-42f2-bee6-9bac5c001788 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:16.237628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d1c6b3a-1d26-42f2-bee6-9bac5c001788 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:16.238093 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d1c6b3a-1d26-42f2-bee6-9bac5c001788 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:16.360921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6e270575-3a08-47e2-b09e-fa54d2c8a9e3 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:16.368656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6e270575-3a08-47e2-b09e-fa54d2c8a9e3 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1313 microversion: 2.1 time: 0.151153 Jun 10 13:08:16.369254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 231/449] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:16 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1313 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:16.369702 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3d1c6b3a-1d26-42f2-bee6-9bac5c001788 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:16.373861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9272627f-a557-41df-bc08-0bfa59401e12 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:16.374971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9272627f-a557-41df-bc08-0bfa59401e12 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:16.378897 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3d1c6b3a-1d26-42f2-bee6-9bac5c001788 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:16.380372 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3d1c6b3a-1d26-42f2-bee6-9bac5c001788 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:16.380638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9272627f-a557-41df-bc08-0bfa59401e12 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:16.381121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9272627f-a557-41df-bc08-0bfa59401e12 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:16.381661 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3d1c6b3a-1d26-42f2-bee6-9bac5c001788 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.156574 Jun 10 13:08:16.381881 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9272627f-a557-41df-bc08-0bfa59401e12 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:16.382213 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 219/450] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:16 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:16.382362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9272627f-a557-41df-bc08-0bfa59401e12 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:16.399825 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-55db7e7f-0eec-4f7a-b359-0994247c9a2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:16.400427 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-55db7e7f-0eec-4f7a-b359-0994247c9a2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:16.406932 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-55db7e7f-0eec-4f7a-b359-0994247c9a2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:16.407360 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-55db7e7f-0eec-4f7a-b359-0994247c9a2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:16.408116 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-55db7e7f-0eec-4f7a-b359-0994247c9a2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:16.408547 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-55db7e7f-0eec-4f7a-b359-0994247c9a2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:16.441875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9272627f-a557-41df-bc08-0bfa59401e12 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:16.449331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9272627f-a557-41df-bc08-0bfa59401e12 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1194 microversion: 2.1 time: 0.079104 Jun 10 13:08:16.449914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 232/451] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:16 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1194 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:16.591903 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-55db7e7f-0eec-4f7a-b359-0994247c9a2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:16.603440 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-55db7e7f-0eec-4f7a-b359-0994247c9a2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.207068 Jun 10 13:08:16.604333 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 220/452] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:16 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:17.047899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2600aaef-197e-445a-9673-51f8703777a1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:17.048450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2600aaef-197e-445a-9673-51f8703777a1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:17.054838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2600aaef-197e-445a-9673-51f8703777a1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:17.055277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2600aaef-197e-445a-9673-51f8703777a1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:17.056080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2600aaef-197e-445a-9673-51f8703777a1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:17.056510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2600aaef-197e-445a-9673-51f8703777a1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:17.081869 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2600aaef-197e-445a-9673-51f8703777a1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:08:17.099117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:17.319975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2600aaef-197e-445a-9673-51f8703777a1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "DELETE /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 204 len: 0 microversion: 2.1 time: 0.275578 Jun 10 13:08:17.320948 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 233/453] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:08:17 2022] DELETE /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jun 10 13:08:17.330365 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5511b2d1-17eb-40c6-8060-9c3eecffb7a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:17.331541 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5511b2d1-17eb-40c6-8060-9c3eecffb7a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:17.344862 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5511b2d1-17eb-40c6-8060-9c3eecffb7a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:17.344862 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5511b2d1-17eb-40c6-8060-9c3eecffb7a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:17.344862 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5511b2d1-17eb-40c6-8060-9c3eecffb7a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:17.344862 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5511b2d1-17eb-40c6-8060-9c3eecffb7a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:17.386468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7d594e9f-fd21-4c3f-b5b1-684cd8b13514 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:17.387304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7d594e9f-fd21-4c3f-b5b1-684cd8b13514 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:17.394026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d594e9f-fd21-4c3f-b5b1-684cd8b13514 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:17.394421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d594e9f-fd21-4c3f-b5b1-684cd8b13514 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:17.395186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d594e9f-fd21-4c3f-b5b1-684cd8b13514 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:17.395619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d594e9f-fd21-4c3f-b5b1-684cd8b13514 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:17.467257 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5511b2d1-17eb-40c6-8060-9c3eecffb7a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:17.474785 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5511b2d1-17eb-40c6-8060-9c3eecffb7a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1515 microversion: 2.1 time: 0.149157 Jun 10 13:08:17.475356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 221/454] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:17 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1515 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:17.479902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e7c2015c-1bb4-4d17-8fcd-4e94126bbc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:17.480509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e7c2015c-1bb4-4d17-8fcd-4e94126bbc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:17.487210 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e7c2015c-1bb4-4d17-8fcd-4e94126bbc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:17.487889 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e7c2015c-1bb4-4d17-8fcd-4e94126bbc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:17.488662 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e7c2015c-1bb4-4d17-8fcd-4e94126bbc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:17.489129 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e7c2015c-1bb4-4d17-8fcd-4e94126bbc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:17.524271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7d594e9f-fd21-4c3f-b5b1-684cd8b13514 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:17.533364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7d594e9f-fd21-4c3f-b5b1-684cd8b13514 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1497 microversion: 2.1 time: 0.151087 Jun 10 13:08:17.534072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 234/455] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:17 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:17.538630 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a38e80b2-46b7-4ae2-bfca-fceee78f7eb7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:17.539546 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a38e80b2-46b7-4ae2-bfca-fceee78f7eb7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:17.548282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a38e80b2-46b7-4ae2-bfca-fceee78f7eb7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:17.548927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a38e80b2-46b7-4ae2-bfca-fceee78f7eb7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:17.549791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a38e80b2-46b7-4ae2-bfca-fceee78f7eb7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:17.551125 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a38e80b2-46b7-4ae2-bfca-fceee78f7eb7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:17.623172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a38e80b2-46b7-4ae2-bfca-fceee78f7eb7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:17.631730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a38e80b2-46b7-4ae2-bfca-fceee78f7eb7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1204 microversion: 2.1 time: 0.096391 Jun 10 13:08:17.632415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 235/456] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:17 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1204 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:17.638359 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-af5558b4-3545-451a-8430-02048414fb7e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:17.638961 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-af5558b4-3545-451a-8430-02048414fb7e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:17.644251 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e7c2015c-1bb4-4d17-8fcd-4e94126bbc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:17.645418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-af5558b4-3545-451a-8430-02048414fb7e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:17.645874 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-af5558b4-3545-451a-8430-02048414fb7e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:17.646675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-af5558b4-3545-451a-8430-02048414fb7e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:17.647113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-af5558b4-3545-451a-8430-02048414fb7e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:17.655377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e7c2015c-1bb4-4d17-8fcd-4e94126bbc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:17.657121 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e7c2015c-1bb4-4d17-8fcd-4e94126bbc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:17.658311 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e7c2015c-1bb4-4d17-8fcd-4e94126bbc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.181948 Jun 10 13:08:17.658869 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 222/457] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:17 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:08:17.787747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-af5558b4-3545-451a-8430-02048414fb7e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:17.795494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-af5558b4-3545-451a-8430-02048414fb7e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.160952 Jun 10 13:08:17.796056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 236/458] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:17 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:18.470242 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b5a8df1d-153a-4fc3-b80f-f498302572ec tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:18.470767 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b5a8df1d-153a-4fc3-b80f-f498302572ec tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:18.477363 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b5a8df1d-153a-4fc3-b80f-f498302572ec tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:18.477964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b5a8df1d-153a-4fc3-b80f-f498302572ec tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:18.478703 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b5a8df1d-153a-4fc3-b80f-f498302572ec tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:18.479140 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b5a8df1d-153a-4fc3-b80f-f498302572ec tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:18.491576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6ba6f144-dd2c-4de8-9543-f7e774a24bee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:18.492172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6ba6f144-dd2c-4de8-9543-f7e774a24bee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:18.499116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ba6f144-dd2c-4de8-9543-f7e774a24bee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:18.499564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ba6f144-dd2c-4de8-9543-f7e774a24bee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:18.500326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ba6f144-dd2c-4de8-9543-f7e774a24bee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:18.500990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ba6f144-dd2c-4de8-9543-f7e774a24bee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:18.506724 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b5a8df1d-153a-4fc3-b80f-f498302572ec tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Going to try to terminate instance {{(pid=118093) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:08:18.652830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6ba6f144-dd2c-4de8-9543-f7e774a24bee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:18.658788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6ba6f144-dd2c-4de8-9543-f7e774a24bee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1515 microversion: 2.1 time: 0.171046 Jun 10 13:08:18.659354 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 237/459] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:18 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1515 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:18.672803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e364175d-aedc-4142-8ce1-f50663d0140a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:18.673426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e364175d-aedc-4142-8ce1-f50663d0140a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:18.679401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e364175d-aedc-4142-8ce1-f50663d0140a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:18.683488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e364175d-aedc-4142-8ce1-f50663d0140a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:18.683488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e364175d-aedc-4142-8ce1-f50663d0140a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:18.683488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e364175d-aedc-4142-8ce1-f50663d0140a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:18.746402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e364175d-aedc-4142-8ce1-f50663d0140a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:18.754296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e364175d-aedc-4142-8ce1-f50663d0140a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1204 microversion: 2.1 time: 0.085202 Jun 10 13:08:18.754859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 238/460] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:18 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1204 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:18.759261 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-92c24eb3-7b47-41bc-8f59-d3701974d22d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:18.759921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-92c24eb3-7b47-41bc-8f59-d3701974d22d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:18.765769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-92c24eb3-7b47-41bc-8f59-d3701974d22d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:18.767268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-92c24eb3-7b47-41bc-8f59-d3701974d22d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:18.767770 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-92c24eb3-7b47-41bc-8f59-d3701974d22d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:18.768193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-92c24eb3-7b47-41bc-8f59-d3701974d22d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:18.789362 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b5a8df1d-153a-4fc3-b80f-f498302572ec tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "DELETE /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 204 len: 0 microversion: 2.1 time: 0.323054 Jun 10 13:08:18.789972 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 223/461] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:08:18 2022] DELETE /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jun 10 13:08:18.798813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d2334330-4198-4b39-a708-c15eb58866b5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:18.799515 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d2334330-4198-4b39-a708-c15eb58866b5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:18.807879 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d2334330-4198-4b39-a708-c15eb58866b5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:18.807879 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d2334330-4198-4b39-a708-c15eb58866b5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:18.808477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d2334330-4198-4b39-a708-c15eb58866b5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:18.808963 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d2334330-4198-4b39-a708-c15eb58866b5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:18.929820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-92c24eb3-7b47-41bc-8f59-d3701974d22d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:18.957109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-92c24eb3-7b47-41bc-8f59-d3701974d22d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:18.959257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-92c24eb3-7b47-41bc-8f59-d3701974d22d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:18.960379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-92c24eb3-7b47-41bc-8f59-d3701974d22d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.204424 Jun 10 13:08:18.961153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 239/462] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:18 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:08:18.968316 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-96089ad1-762f-44a9-8940-78cdd3d98cf4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:18.968637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-96089ad1-762f-44a9-8940-78cdd3d98cf4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:18.971695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:18.976726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96089ad1-762f-44a9-8940-78cdd3d98cf4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:18.977212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96089ad1-762f-44a9-8940-78cdd3d98cf4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:18.977991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96089ad1-762f-44a9-8940-78cdd3d98cf4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:18.978422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96089ad1-762f-44a9-8940-78cdd3d98cf4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:19.007395 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d2334330-4198-4b39-a708-c15eb58866b5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:19.014212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d2334330-4198-4b39-a708-c15eb58866b5 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1503 microversion: 2.1 time: 0.219499 Jun 10 13:08:19.019422 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 224/463] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:18 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1503 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:19.117747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-96089ad1-762f-44a9-8940-78cdd3d98cf4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:19.123904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-96089ad1-762f-44a9-8940-78cdd3d98cf4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1501 microversion: 2.1 time: 0.161805 Jun 10 13:08:19.124478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 240/464] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:18 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1501 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:19.140738 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e832b7e6-7409-4b40-a4a2-3208c66f5b57 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "097f4b85-9dd0-4ebd-bc2a-2840770e350c"}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:19.141915 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e832b7e6-7409-4b40-a4a2-3208c66f5b57 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:19.147911 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e832b7e6-7409-4b40-a4a2-3208c66f5b57 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:19.148351 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e832b7e6-7409-4b40-a4a2-3208c66f5b57 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:19.149141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e832b7e6-7409-4b40-a4a2-3208c66f5b57 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:19.149568 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e832b7e6-7409-4b40-a4a2-3208c66f5b57 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:19.153322 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:19.171548 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.wsgi [None req-e832b7e6-7409-4b40-a4a2-3208c66f5b57 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] HTTP exception thrown: Cannot 'attach_volume' instance 5d205551-9f23-46f8-b67b-ca5bafd58182 while it is in vm_state rescued Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e832b7e6-7409-4b40-a4a2-3208c66f5b57 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Request method failure captured: Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: request: POST /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182/os-volume_attachments HTTP/1.1 Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept: application/json Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept-Encoding: identity Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Connection: close Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Length: 74 Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Type: application/json Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Host: 10.222.0.25 Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Openstack-System-Scope: None Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: User-Agent: python-urllib3/1.26.9 Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Auth-Token: gAAAAABio0HbeUNf0EPZ-aslwQVWNd8-Z8OHNvp8Sb8m5gjvFM0tL0pvFj03ha5wm_h98nRGWdTUfHXBkNJa9XdKTFzGaRbJsRxJn5YRVRoYaqROYy-SpZpzuBlLa9e8KDT13WQC8G8gU0qsOwDmzLx6BmD9PCuaOyoARKjO1sMu6-pWpQAV11w Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Id: None Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Name: None Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Identity-Status: Confirmed Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Is-Admin-Project: True Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Id: default Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Name: Default Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Id: 03423e3482614374afa8c929bd931262 Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-696024477 Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Role: reader,member Jun 10 13:08:19.173763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Roles: reader,member Jun 10 13:08:19.175326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/03423e3482614374afa8c929bd931262"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/03423e3482614374afa8c929bd931262"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_03423e3482614374afa8c929bd931262", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/03423e3482614374afa8c929bd931262"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:08:19.175326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant: tempest-ServerRescueNegativeTestJSON-696024477 Jun 10 13:08:19.175326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Id: 03423e3482614374afa8c929bd931262 Jun 10 13:08:19.175326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-696024477 Jun 10 13:08:19.175326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User: tempest-ServerRescueNegativeTestJSON-696024477-project Jun 10 13:08:19.175326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Id: default Jun 10 13:08:19.175326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Name: Default Jun 10 13:08:19.175326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Id: 25f73d0bd4e64057a922d11be46eb04c Jun 10 13:08:19.175326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Name: tempest-ServerRescueNegativeTestJSON-696024477-project Jun 10 13:08:19.175326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Jun 10 13:08:19.175326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: {"volumeAttachment": {"volumeId": "097f4b85-9dd0-4ebd-bc2a-2840770e350c"}} Jun 10 13:08:19.175326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: method: > Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: exception: Cannot 'attach_volume' instance 5d205551-9f23-46f8-b67b-ca5bafd58182 while it is in vm_state rescued Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: action_args: {'server_id': '5d205551-9f23-46f8-b67b-ca5bafd58182', 'body': {'volumeAttachment': {'volumeId': '097f4b85-9dd0-4ebd-bc2a-2840770e350c'}}} Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 5d205551-9f23-46f8-b67b-ca5bafd58182 in vm_state rescued. Cannot attach_volume while the instance is in this state. Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:08:19.176228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 5d205551-9f23-46f8-b67b-ca5bafd58182 while it is in vm_state rescued Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:08:19.177677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:08:19.179074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:08:19.179074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:08:19.179074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 5d205551-9f23-46f8-b67b-ca5bafd58182 while it is in vm_state rescued Jun 10 13:08:19.179074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:08:19.179074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e832b7e6-7409-4b40-a4a2-3208c66f5b57 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Returning 409 to user: Cannot 'attach_volume' instance 5d205551-9f23-46f8-b67b-ca5bafd58182 while it is in vm_state rescued {{(pid=118093) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:08:19.179074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e832b7e6-7409-4b40-a4a2-3208c66f5b57 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "POST /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.038313 Jun 10 13:08:19.179074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 225/465] 10.222.0.25 () {60 vars in 1322 bytes} [Fri Jun 10 13:08:19 2022] POST /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182/os-volume_attachments => generated 152 bytes in 39 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jun 10 13:08:19.185506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4a4054a0-00d9-405e-98a9-6efd9ab3f4ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:19.185506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4a4054a0-00d9-405e-98a9-6efd9ab3f4ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:19.192003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a4054a0-00d9-405e-98a9-6efd9ab3f4ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:19.192442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a4054a0-00d9-405e-98a9-6efd9ab3f4ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:19.193228 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a4054a0-00d9-405e-98a9-6efd9ab3f4ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:19.193661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a4054a0-00d9-405e-98a9-6efd9ab3f4ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:19.278300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-23db34ef-2878-468c-9c83-305c57739227 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:19.278782 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-23db34ef-2878-468c-9c83-305c57739227 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:19.286018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-23db34ef-2878-468c-9c83-305c57739227 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:19.286461 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-23db34ef-2878-468c-9c83-305c57739227 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:19.287206 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-23db34ef-2878-468c-9c83-305c57739227 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:19.287629 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-23db34ef-2878-468c-9c83-305c57739227 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:19.368681 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-23db34ef-2878-468c-9c83-305c57739227 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lazy-loading 'services' on Instance uuid 9e403cf8-5aee-49f3-bbbc-600263bd148a {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:08:19.398834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4a4054a0-00d9-405e-98a9-6efd9ab3f4ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "POST /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182/action" status: 202 len: 0 microversion: 2.1 time: 0.218547 Jun 10 13:08:19.402482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 241/466] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Jun 10 13:08:19 2022] POST /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182/action => generated 0 bytes in 220 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jun 10 13:08:19.408214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-544a8cb8-4ec8-42bc-9a6d-5c2ee19c92cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:19.409322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-544a8cb8-4ec8-42bc-9a6d-5c2ee19c92cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:19.415070 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-544a8cb8-4ec8-42bc-9a6d-5c2ee19c92cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:19.415513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-544a8cb8-4ec8-42bc-9a6d-5c2ee19c92cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:19.416276 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-544a8cb8-4ec8-42bc-9a6d-5c2ee19c92cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:19.416711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-544a8cb8-4ec8-42bc-9a6d-5c2ee19c92cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:19.546395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-544a8cb8-4ec8-42bc-9a6d-5c2ee19c92cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:19.554580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-544a8cb8-4ec8-42bc-9a6d-5c2ee19c92cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1509 microversion: 2.1 time: 0.150812 Jun 10 13:08:19.555129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 242/467] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:19 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1509 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:19.566269 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-23db34ef-2878-468c-9c83-305c57739227 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "DELETE /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/os-volume_attachments/6b8950d0-dd0c-47fe-a7bd-d91571ff68ba" status: 202 len: 0 microversion: 2.1 time: 0.291609 Jun 10 13:08:19.566858 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 226/468] 10.222.0.25 () {58 vars in 1415 bytes} [Fri Jun 10 13:08:19 2022] DELETE /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/os-volume_attachments/6b8950d0-dd0c-47fe-a7bd-d91571ff68ba => generated 0 bytes in 293 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:08:19.675622 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1411085c-fada-4af8-9773-02cad2eefb60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:19.676171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1411085c-fada-4af8-9773-02cad2eefb60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:19.682175 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1411085c-fada-4af8-9773-02cad2eefb60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:19.682626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1411085c-fada-4af8-9773-02cad2eefb60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:19.683392 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1411085c-fada-4af8-9773-02cad2eefb60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:19.683828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1411085c-fada-4af8-9773-02cad2eefb60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:19.772173 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3234dea0-ae92-4072-9dc1-086c318e0a5d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:19.772913 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3234dea0-ae92-4072-9dc1-086c318e0a5d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:19.779329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3234dea0-ae92-4072-9dc1-086c318e0a5d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:19.779774 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3234dea0-ae92-4072-9dc1-086c318e0a5d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:19.780540 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3234dea0-ae92-4072-9dc1-086c318e0a5d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:19.780996 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3234dea0-ae92-4072-9dc1-086c318e0a5d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:19.815390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1411085c-fada-4af8-9773-02cad2eefb60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:19.823141 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1411085c-fada-4af8-9773-02cad2eefb60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1515 microversion: 2.1 time: 0.151305 Jun 10 13:08:19.823910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 243/469] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:19 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1515 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:19.915431 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3234dea0-ae92-4072-9dc1-086c318e0a5d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:19.923027 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3234dea0-ae92-4072-9dc1-086c318e0a5d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1272 microversion: 2.1 time: 0.155121 Jun 10 13:08:19.923609 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 227/470] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:19 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1272 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:19.978072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-20a0702c-77f0-4a1a-a974-625d52b65e23 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:19.978701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-20a0702c-77f0-4a1a-a974-625d52b65e23 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:19.984689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20a0702c-77f0-4a1a-a974-625d52b65e23 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:19.985236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20a0702c-77f0-4a1a-a974-625d52b65e23 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:19.986016 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20a0702c-77f0-4a1a-a974-625d52b65e23 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:19.986735 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20a0702c-77f0-4a1a-a974-625d52b65e23 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:20.033816 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e303d427-e21f-4c3e-8c38-123d12604d15 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:20.034611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e303d427-e21f-4c3e-8c38-123d12604d15 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:20.040855 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e303d427-e21f-4c3e-8c38-123d12604d15 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:20.041298 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e303d427-e21f-4c3e-8c38-123d12604d15 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:20.042060 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e303d427-e21f-4c3e-8c38-123d12604d15 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:20.042481 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e303d427-e21f-4c3e-8c38-123d12604d15 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:20.115470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-20a0702c-77f0-4a1a-a974-625d52b65e23 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:20.124094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-20a0702c-77f0-4a1a-a974-625d52b65e23 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:20.125613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-20a0702c-77f0-4a1a-a974-625d52b65e23 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:20.127320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-20a0702c-77f0-4a1a-a974-625d52b65e23 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.153109 Jun 10 13:08:20.127874 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 244/471] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:19 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:20.169759 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e303d427-e21f-4c3e-8c38-123d12604d15 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:20.177063 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e303d427-e21f-4c3e-8c38-123d12604d15 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1503 microversion: 2.1 time: 0.149506 Jun 10 13:08:20.177622 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 228/472] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:20 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1503 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:20.571691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4a4dffe9-e8b3-4628-bfc0-8d72edf6b9b6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:20.572286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4a4dffe9-e8b3-4628-bfc0-8d72edf6b9b6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:20.579285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a4dffe9-e8b3-4628-bfc0-8d72edf6b9b6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:20.579725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a4dffe9-e8b3-4628-bfc0-8d72edf6b9b6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:20.580472 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a4dffe9-e8b3-4628-bfc0-8d72edf6b9b6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:20.580928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a4dffe9-e8b3-4628-bfc0-8d72edf6b9b6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:20.710386 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4a4dffe9-e8b3-4628-bfc0-8d72edf6b9b6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:20.717811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4a4dffe9-e8b3-4628-bfc0-8d72edf6b9b6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1509 microversion: 2.1 time: 0.149931 Jun 10 13:08:20.718376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 245/473] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:20 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1509 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:20.840478 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-54a505ea-fec1-4f07-aa3f-5e560a851a18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:20.841112 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-54a505ea-fec1-4f07-aa3f-5e560a851a18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:20.848142 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-54a505ea-fec1-4f07-aa3f-5e560a851a18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:20.848587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-54a505ea-fec1-4f07-aa3f-5e560a851a18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:20.849377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-54a505ea-fec1-4f07-aa3f-5e560a851a18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:20.849829 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-54a505ea-fec1-4f07-aa3f-5e560a851a18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:20.940028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2c8b4cbe-b773-4b18-92b9-04dfe1591af1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:20.940722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2c8b4cbe-b773-4b18-92b9-04dfe1591af1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:20.946779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2c8b4cbe-b773-4b18-92b9-04dfe1591af1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:20.947291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2c8b4cbe-b773-4b18-92b9-04dfe1591af1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:20.948056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2c8b4cbe-b773-4b18-92b9-04dfe1591af1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:20.948483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2c8b4cbe-b773-4b18-92b9-04dfe1591af1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:20.977201 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-54a505ea-fec1-4f07-aa3f-5e560a851a18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:20.984685 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-54a505ea-fec1-4f07-aa3f-5e560a851a18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1515 microversion: 2.1 time: 0.148092 Jun 10 13:08:20.985944 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 229/474] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:20 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1515 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:21.079522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2c8b4cbe-b773-4b18-92b9-04dfe1591af1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:21.086884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2c8b4cbe-b773-4b18-92b9-04dfe1591af1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1272 microversion: 2.1 time: 0.150782 Jun 10 13:08:21.087438 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 246/475] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:20 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1272 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:21.145909 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ed7dd5f2-5627-4732-b855-867f54589ffa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:21.146511 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ed7dd5f2-5627-4732-b855-867f54589ffa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:21.152494 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ed7dd5f2-5627-4732-b855-867f54589ffa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:21.152971 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ed7dd5f2-5627-4732-b855-867f54589ffa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:21.153725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ed7dd5f2-5627-4732-b855-867f54589ffa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:21.154157 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ed7dd5f2-5627-4732-b855-867f54589ffa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:21.193817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c8bb2dcf-1538-475a-be21-252807b56084 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:21.194425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c8bb2dcf-1538-475a-be21-252807b56084 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:21.200363 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c8bb2dcf-1538-475a-be21-252807b56084 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:21.200826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c8bb2dcf-1538-475a-be21-252807b56084 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:21.201621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c8bb2dcf-1538-475a-be21-252807b56084 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:21.202052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c8bb2dcf-1538-475a-be21-252807b56084 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:21.313457 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ed7dd5f2-5627-4732-b855-867f54589ffa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:21.322318 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ed7dd5f2-5627-4732-b855-867f54589ffa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:21.324150 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ed7dd5f2-5627-4732-b855-867f54589ffa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:21.325388 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ed7dd5f2-5627-4732-b855-867f54589ffa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.183856 Jun 10 13:08:21.326293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 230/476] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:21 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:08:21.335362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c8bb2dcf-1538-475a-be21-252807b56084 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:21.345137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c8bb2dcf-1538-475a-be21-252807b56084 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1503 microversion: 2.1 time: 0.154046 Jun 10 13:08:21.345317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 247/477] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:21 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1503 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:21.734756 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3fbd0d44-756e-452b-be6f-f0c3874133a0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:21.735363 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3fbd0d44-756e-452b-be6f-f0c3874133a0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:21.741663 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3fbd0d44-756e-452b-be6f-f0c3874133a0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:21.742113 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3fbd0d44-756e-452b-be6f-f0c3874133a0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:21.742874 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3fbd0d44-756e-452b-be6f-f0c3874133a0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:21.743307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3fbd0d44-756e-452b-be6f-f0c3874133a0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:21.971616 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3fbd0d44-756e-452b-be6f-f0c3874133a0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:21.991441 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3fbd0d44-756e-452b-be6f-f0c3874133a0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1509 microversion: 2.1 time: 0.260453 Jun 10 13:08:21.992034 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 231/478] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:21 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1509 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:22.003076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9128d99e-3ae8-48ff-b79c-4790207d1200 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:22.008686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9128d99e-3ae8-48ff-b79c-4790207d1200 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:22.010423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9128d99e-3ae8-48ff-b79c-4790207d1200 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:22.010881 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9128d99e-3ae8-48ff-b79c-4790207d1200 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:22.011701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9128d99e-3ae8-48ff-b79c-4790207d1200 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:22.012049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9128d99e-3ae8-48ff-b79c-4790207d1200 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:22.016227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:22.103440 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-652ec091-07c3-4ee4-91a7-dc47be4fee76 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:22.104051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-652ec091-07c3-4ee4-91a7-dc47be4fee76 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:22.113347 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-652ec091-07c3-4ee4-91a7-dc47be4fee76 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:22.114065 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-652ec091-07c3-4ee4-91a7-dc47be4fee76 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:22.114848 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-652ec091-07c3-4ee4-91a7-dc47be4fee76 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:22.115282 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-652ec091-07c3-4ee4-91a7-dc47be4fee76 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:22.156259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9128d99e-3ae8-48ff-b79c-4790207d1200 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:22.164281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9128d99e-3ae8-48ff-b79c-4790207d1200 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1515 microversion: 2.1 time: 0.165682 Jun 10 13:08:22.164879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 248/479] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:21 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1515 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:22.247119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-ac3f2ff5-e63f-4bc7-bd84-9ee86711c947 req-8e44670a-1d00-4004-a8d3-13af4ef9ab1d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3baaefc5-1a36-4fa6-88d9-1764e5631a32", "tag": "72eeacbd-3288-4d28-90e4-93e208b2d17d"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:22.256523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-ac3f2ff5-e63f-4bc7-bd84-9ee86711c947 req-8e44670a-1d00-4004-a8d3-13af4ef9ab1d service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:22.256983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-ac3f2ff5-e63f-4bc7-bd84-9ee86711c947 req-8e44670a-1d00-4004-a8d3-13af4ef9ab1d service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:22.268297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-ac3f2ff5-e63f-4bc7-bd84-9ee86711c947 req-8e44670a-1d00-4004-a8d3-13af4ef9ab1d service nova] Creating event network-changed:72eeacbd-3288-4d28-90e4-93e208b2d17d for instance 3baaefc5-1a36-4fa6-88d9-1764e5631a32 on hv-cinder-maste Jun 10 13:08:22.274932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-ac3f2ff5-e63f-4bc7-bd84-9ee86711c947 req-8e44670a-1d00-4004-a8d3-13af4ef9ab1d service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029485 Jun 10 13:08:22.274932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 249/480] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:08:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:08:22.314311 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-652ec091-07c3-4ee4-91a7-dc47be4fee76 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:22.322531 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-652ec091-07c3-4ee4-91a7-dc47be4fee76 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1272 microversion: 2.1 time: 0.222700 Jun 10 13:08:22.332064 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 232/481] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:22 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1272 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:22.344088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-659e5f4c-5b08-4fa4-be78-4bb78183e625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:22.345468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-659e5f4c-5b08-4fa4-be78-4bb78183e625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:22.352005 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-659e5f4c-5b08-4fa4-be78-4bb78183e625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:22.352533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-659e5f4c-5b08-4fa4-be78-4bb78183e625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:22.353410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-659e5f4c-5b08-4fa4-be78-4bb78183e625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:22.353917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-659e5f4c-5b08-4fa4-be78-4bb78183e625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:22.361537 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4da77122-6997-425d-a4e7-3bc5dcfb9218 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:22.362386 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4da77122-6997-425d-a4e7-3bc5dcfb9218 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:22.368901 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4da77122-6997-425d-a4e7-3bc5dcfb9218 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:22.372369 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4da77122-6997-425d-a4e7-3bc5dcfb9218 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:22.372369 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4da77122-6997-425d-a4e7-3bc5dcfb9218 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:22.372369 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4da77122-6997-425d-a4e7-3bc5dcfb9218 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:22.493242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-659e5f4c-5b08-4fa4-be78-4bb78183e625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:22.499692 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4da77122-6997-425d-a4e7-3bc5dcfb9218 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:22.502138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-659e5f4c-5b08-4fa4-be78-4bb78183e625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:22.503647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-659e5f4c-5b08-4fa4-be78-4bb78183e625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:22.504821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-659e5f4c-5b08-4fa4-be78-4bb78183e625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.164972 Jun 10 13:08:22.505378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 250/482] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:22 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:08:22.507240 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4da77122-6997-425d-a4e7-3bc5dcfb9218 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1503 microversion: 2.1 time: 0.150406 Jun 10 13:08:22.507790 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 233/483] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:22 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1503 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:23.009349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dfd8bc6d-f3ca-4d13-beb6-e0dcb4b12d9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:23.010753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dfd8bc6d-f3ca-4d13-beb6-e0dcb4b12d9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:23.017536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dfd8bc6d-f3ca-4d13-beb6-e0dcb4b12d9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:23.017985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dfd8bc6d-f3ca-4d13-beb6-e0dcb4b12d9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:23.018743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dfd8bc6d-f3ca-4d13-beb6-e0dcb4b12d9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:23.019177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dfd8bc6d-f3ca-4d13-beb6-e0dcb4b12d9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:23.182402 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d1508a9a-4110-4be0-b69e-e5f568ba3f7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:23.185913 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d1508a9a-4110-4be0-b69e-e5f568ba3f7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:23.190646 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d1508a9a-4110-4be0-b69e-e5f568ba3f7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:23.191099 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d1508a9a-4110-4be0-b69e-e5f568ba3f7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:23.191873 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d1508a9a-4110-4be0-b69e-e5f568ba3f7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:23.192302 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d1508a9a-4110-4be0-b69e-e5f568ba3f7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:23.241570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dfd8bc6d-f3ca-4d13-beb6-e0dcb4b12d9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:23.249625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dfd8bc6d-f3ca-4d13-beb6-e0dcb4b12d9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1509 microversion: 2.1 time: 0.244111 Jun 10 13:08:23.250512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 251/484] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:23 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1509 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:23.341036 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a9368103-6909-4651-94a9-a3bff9a379eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:23.341901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a9368103-6909-4651-94a9-a3bff9a379eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:23.349116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a9368103-6909-4651-94a9-a3bff9a379eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:23.349566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a9368103-6909-4651-94a9-a3bff9a379eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:23.350343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a9368103-6909-4651-94a9-a3bff9a379eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:23.350778 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a9368103-6909-4651-94a9-a3bff9a379eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:23.522292 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d1508a9a-4110-4be0-b69e-e5f568ba3f7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:23.527648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a9368103-6909-4651-94a9-a3bff9a379eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:23.530024 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d1508a9a-4110-4be0-b69e-e5f568ba3f7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1515 microversion: 2.1 time: 0.351690 Jun 10 13:08:23.530600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 234/485] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:23 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1515 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:23.535262 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1d35b5fa-930b-4d67-8b02-ea3fbcdbb76a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:23.536097 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1d35b5fa-930b-4d67-8b02-ea3fbcdbb76a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:23.542843 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1d35b5fa-930b-4d67-8b02-ea3fbcdbb76a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:23.543500 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1d35b5fa-930b-4d67-8b02-ea3fbcdbb76a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:23.544270 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1d35b5fa-930b-4d67-8b02-ea3fbcdbb76a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:23.544702 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1d35b5fa-930b-4d67-8b02-ea3fbcdbb76a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:23.545118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a9368103-6909-4651-94a9-a3bff9a379eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1272 microversion: 2.1 time: 0.209399 Jun 10 13:08:23.545708 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 252/486] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:23 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1272 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:23.551381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a933cf84-1be4-4922-b5bf-5f3cb6c296db tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:23.552150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a933cf84-1be4-4922-b5bf-5f3cb6c296db tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:23.558460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a933cf84-1be4-4922-b5bf-5f3cb6c296db tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:23.558917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a933cf84-1be4-4922-b5bf-5f3cb6c296db tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:23.559822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a933cf84-1be4-4922-b5bf-5f3cb6c296db tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:23.560265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a933cf84-1be4-4922-b5bf-5f3cb6c296db tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:23.753533 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1d35b5fa-930b-4d67-8b02-ea3fbcdbb76a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:23.762549 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1d35b5fa-930b-4d67-8b02-ea3fbcdbb76a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:23.763953 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1d35b5fa-930b-4d67-8b02-ea3fbcdbb76a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:23.765777 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1d35b5fa-930b-4d67-8b02-ea3fbcdbb76a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.234064 Jun 10 13:08:23.774260 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 235/487] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:23 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:08:23.840091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a933cf84-1be4-4922-b5bf-5f3cb6c296db tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:23.848156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a933cf84-1be4-4922-b5bf-5f3cb6c296db tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1503 microversion: 2.1 time: 0.301286 Jun 10 13:08:23.848722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 253/488] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:23 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1503 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:24.271233 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f6515b12-17c9-4454-9156-c200194f7138 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:24.272179 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f6515b12-17c9-4454-9156-c200194f7138 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:24.279825 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6515b12-17c9-4454-9156-c200194f7138 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:24.280367 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6515b12-17c9-4454-9156-c200194f7138 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:24.281286 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6515b12-17c9-4454-9156-c200194f7138 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:24.281880 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6515b12-17c9-4454-9156-c200194f7138 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:24.324873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-92db8632-0724-40a1-a7bb-7f33fca4d752 req-1aee7b77-2050-457f-ae17-e56984453d65 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1d2dd021-a498-436e-8f04-13e3b612d24c", "name": "network-vif-deleted", "tag": "e6ced7a0-efaf-4a5c-a6d4-e23d099b7ed8"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:24.340612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-92db8632-0724-40a1-a7bb-7f33fca4d752 req-1aee7b77-2050-457f-ae17-e56984453d65 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:24.341102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-92db8632-0724-40a1-a7bb-7f33fca4d752 req-1aee7b77-2050-457f-ae17-e56984453d65 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:24.353422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-92db8632-0724-40a1-a7bb-7f33fca4d752 req-1aee7b77-2050-457f-ae17-e56984453d65 service nova] Creating event network-vif-deleted:e6ced7a0-efaf-4a5c-a6d4-e23d099b7ed8 for instance 1d2dd021-a498-436e-8f04-13e3b612d24c on hv-cinder-maste Jun 10 13:08:24.358227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-92db8632-0724-40a1-a7bb-7f33fca4d752 req-1aee7b77-2050-457f-ae17-e56984453d65 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037131 Jun 10 13:08:24.358820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 254/489] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:08:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:08:24.547879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dcad2404-fee5-4c13-b581-bdc26f7096ce tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:24.549034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dcad2404-fee5-4c13-b581-bdc26f7096ce tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:24.555458 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcad2404-fee5-4c13-b581-bdc26f7096ce tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:24.556107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcad2404-fee5-4c13-b581-bdc26f7096ce tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:24.557992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcad2404-fee5-4c13-b581-bdc26f7096ce tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:24.558521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcad2404-fee5-4c13-b581-bdc26f7096ce tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:24.617445 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f6515b12-17c9-4454-9156-c200194f7138 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:24.633718 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f6515b12-17c9-4454-9156-c200194f7138 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1509 microversion: 2.1 time: 0.367117 Jun 10 13:08:24.636034 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 236/490] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:24 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1509 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:24.641952 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e1c608bb-5567-4f04-9c61-9b7a12b52a9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:24.643601 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e1c608bb-5567-4f04-9c61-9b7a12b52a9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:24.650438 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e1c608bb-5567-4f04-9c61-9b7a12b52a9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:24.651157 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e1c608bb-5567-4f04-9c61-9b7a12b52a9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:24.652175 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e1c608bb-5567-4f04-9c61-9b7a12b52a9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:24.652842 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e1c608bb-5567-4f04-9c61-9b7a12b52a9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:24.727408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dcad2404-fee5-4c13-b581-bdc26f7096ce tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:24.735453 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dcad2404-fee5-4c13-b581-bdc26f7096ce tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1515 microversion: 2.1 time: 0.192389 Jun 10 13:08:24.736743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 255/491] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:24 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1515 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:24.784845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f273dc1d-c612-489c-b3b7-e6239700a0b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:24.785453 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f273dc1d-c612-489c-b3b7-e6239700a0b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:24.791586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f273dc1d-c612-489c-b3b7-e6239700a0b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:24.792037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f273dc1d-c612-489c-b3b7-e6239700a0b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:24.793697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f273dc1d-c612-489c-b3b7-e6239700a0b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:24.794138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f273dc1d-c612-489c-b3b7-e6239700a0b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:24.905425 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e1c608bb-5567-4f04-9c61-9b7a12b52a9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:24.917714 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e1c608bb-5567-4f04-9c61-9b7a12b52a9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.280153 Jun 10 13:08:24.919900 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 237/492] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:24 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:24.927883 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b98146a1-e08d-4f3e-a73a-b4951c1a4d68 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:24.929404 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b98146a1-e08d-4f3e-a73a-b4951c1a4d68 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:24.943547 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b98146a1-e08d-4f3e-a73a-b4951c1a4d68 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:24.943999 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b98146a1-e08d-4f3e-a73a-b4951c1a4d68 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:24.944792 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b98146a1-e08d-4f3e-a73a-b4951c1a4d68 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:24.945225 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b98146a1-e08d-4f3e-a73a-b4951c1a4d68 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:24.945560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f273dc1d-c612-489c-b3b7-e6239700a0b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:24.955083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f273dc1d-c612-489c-b3b7-e6239700a0b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:24.956577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f273dc1d-c612-489c-b3b7-e6239700a0b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:24.957848 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f273dc1d-c612-489c-b3b7-e6239700a0b0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.177795 Jun 10 13:08:24.958712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 256/493] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:24 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:24.963757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-78c48f85-cbfd-469f-ad1c-5c5df3849f14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:24.971048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78c48f85-cbfd-469f-ad1c-5c5df3849f14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:24.972801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78c48f85-cbfd-469f-ad1c-5c5df3849f14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:24.984370 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b98146a1-e08d-4f3e-a73a-b4951c1a4d68 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:24.984848 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b98146a1-e08d-4f3e-a73a-b4951c1a4d68 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:24.986013 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-b98146a1-e08d-4f3e-a73a-b4951c1a4d68 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lazy-loading 'fault' on Instance uuid 1d2dd021-a498-436e-8f04-13e3b612d24c {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:08:24.995796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-78c48f85-cbfd-469f-ad1c-5c5df3849f14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Going to try to stop instance {{(pid=118092) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jun 10 13:08:25.030329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b98146a1-e08d-4f3e-a73a-b4951c1a4d68 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:25.038236 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b98146a1-e08d-4f3e-a73a-b4951c1a4d68 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1303 microversion: 2.1 time: 0.117329 Jun 10 13:08:25.038848 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 238/494] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:24 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1303 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:25.180507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-78c48f85-cbfd-469f-ad1c-5c5df3849f14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "POST /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/action" status: 202 len: 0 microversion: 2.1 time: 0.220388 Jun 10 13:08:25.185535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 257/495] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Jun 10 13:08:24 2022] POST /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/action => generated 0 bytes in 222 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jun 10 13:08:25.190075 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9aa3aa4b-bde7-4b36-859f-2c06d4438fd3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:25.190684 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9aa3aa4b-bde7-4b36-859f-2c06d4438fd3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:25.197117 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9aa3aa4b-bde7-4b36-859f-2c06d4438fd3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:25.197571 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9aa3aa4b-bde7-4b36-859f-2c06d4438fd3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:25.198344 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9aa3aa4b-bde7-4b36-859f-2c06d4438fd3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:25.198768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9aa3aa4b-bde7-4b36-859f-2c06d4438fd3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:25.432218 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9aa3aa4b-bde7-4b36-859f-2c06d4438fd3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:25.440359 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9aa3aa4b-bde7-4b36-859f-2c06d4438fd3 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.254687 Jun 10 13:08:25.440947 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 239/496] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:25 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:25.655050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4f306485-8a46-4cb8-b105-dd0b4c2e0f58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:25.655865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4f306485-8a46-4cb8-b105-dd0b4c2e0f58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:25.663026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4f306485-8a46-4cb8-b105-dd0b4c2e0f58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:25.663536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4f306485-8a46-4cb8-b105-dd0b4c2e0f58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:25.664311 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4f306485-8a46-4cb8-b105-dd0b4c2e0f58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:25.664742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4f306485-8a46-4cb8-b105-dd0b4c2e0f58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:25.754384 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-fbd7cdc7-c289-4309-b6a5-6cf5bdcb45c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:25.755002 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-fbd7cdc7-c289-4309-b6a5-6cf5bdcb45c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:25.761362 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fbd7cdc7-c289-4309-b6a5-6cf5bdcb45c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:25.761807 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fbd7cdc7-c289-4309-b6a5-6cf5bdcb45c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:25.762564 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fbd7cdc7-c289-4309-b6a5-6cf5bdcb45c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:25.762988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fbd7cdc7-c289-4309-b6a5-6cf5bdcb45c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:25.833318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4f306485-8a46-4cb8-b105-dd0b4c2e0f58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:25.841419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4f306485-8a46-4cb8-b105-dd0b4c2e0f58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1509 microversion: 2.1 time: 0.191111 Jun 10 13:08:25.842284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 258/497] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:25 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1509 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:25.893829 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-fbd7cdc7-c289-4309-b6a5-6cf5bdcb45c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:25.901741 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-fbd7cdc7-c289-4309-b6a5-6cf5bdcb45c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1304 microversion: 2.1 time: 0.153234 Jun 10 13:08:25.911509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 240/498] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:25 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1304 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:25.935457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3f99a92c-aee2-42a6-9ee7-d46203d150e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:25.936069 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3f99a92c-aee2-42a6-9ee7-d46203d150e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:25.943979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f99a92c-aee2-42a6-9ee7-d46203d150e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:25.944443 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f99a92c-aee2-42a6-9ee7-d46203d150e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:25.945911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f99a92c-aee2-42a6-9ee7-d46203d150e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:25.946672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f99a92c-aee2-42a6-9ee7-d46203d150e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:25.976497 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-340b65b2-f45d-4a22-aa82-e45f32d078e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:25.977197 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-340b65b2-f45d-4a22-aa82-e45f32d078e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:25.983572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-340b65b2-f45d-4a22-aa82-e45f32d078e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:25.984175 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-340b65b2-f45d-4a22-aa82-e45f32d078e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:25.985078 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-340b65b2-f45d-4a22-aa82-e45f32d078e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:25.985612 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-340b65b2-f45d-4a22-aa82-e45f32d078e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:25.990349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:26.090111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f99a92c-aee2-42a6-9ee7-d46203d150e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:26.100794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3f99a92c-aee2-42a6-9ee7-d46203d150e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.169065 Jun 10 13:08:26.101556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 259/499] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:25 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:26.108085 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-37c40bb9-60c5-446f-b283-2bcbce578da0 req-49473f7d-737c-4dea-a825-57d22853973f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "db56898a-dff9-4cad-bf8d-f84b53247fc3", "name": "network-vif-plugged", "status": "completed", "tag": "31e1a9d0-3f1e-4221-a431-639f0e4a6ff4"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:26.117863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-37c40bb9-60c5-446f-b283-2bcbce578da0 req-49473f7d-737c-4dea-a825-57d22853973f service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:26.118452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-37c40bb9-60c5-446f-b283-2bcbce578da0 req-49473f7d-737c-4dea-a825-57d22853973f service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:26.143983 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-340b65b2-f45d-4a22-aa82-e45f32d078e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:26.147215 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-37c40bb9-60c5-446f-b283-2bcbce578da0 req-49473f7d-737c-4dea-a825-57d22853973f service nova] Creating event network-vif-plugged:31e1a9d0-3f1e-4221-a431-639f0e4a6ff4 for instance db56898a-dff9-4cad-bf8d-f84b53247fc3 on hv-cinder-maste Jun 10 13:08:26.152364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-37c40bb9-60c5-446f-b283-2bcbce578da0 req-49473f7d-737c-4dea-a825-57d22853973f service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049176 Jun 10 13:08:26.153089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 260/500] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:08:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:08:26.153710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-340b65b2-f45d-4a22-aa82-e45f32d078e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:26.155514 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-340b65b2-f45d-4a22-aa82-e45f32d078e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:26.156694 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-340b65b2-f45d-4a22-aa82-e45f32d078e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.184494 Jun 10 13:08:26.157510 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 241/501] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:25 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:26.158799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-68019e81-314f-4fd8-998b-6279a3806aeb tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:26.161504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-68019e81-314f-4fd8-998b-6279a3806aeb tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:26.166555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68019e81-314f-4fd8-998b-6279a3806aeb tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:26.167044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68019e81-314f-4fd8-998b-6279a3806aeb tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:26.167773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68019e81-314f-4fd8-998b-6279a3806aeb tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:26.168231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68019e81-314f-4fd8-998b-6279a3806aeb tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:26.200533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68019e81-314f-4fd8-998b-6279a3806aeb tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:26.200533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68019e81-314f-4fd8-998b-6279a3806aeb tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:26.200950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-68019e81-314f-4fd8-998b-6279a3806aeb tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lazy-loading 'fault' on Instance uuid 1d2dd021-a498-436e-8f04-13e3b612d24c {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:08:26.241832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-68019e81-314f-4fd8-998b-6279a3806aeb tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:26.249174 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-68019e81-314f-4fd8-998b-6279a3806aeb tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1303 microversion: 2.1 time: 0.094869 Jun 10 13:08:26.249743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 261/502] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:26 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1303 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:26.390996 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-cea18f36-9284-4de5-8768-89d56162cbe0 req-b6fee393-21b2-4e53-9077-4281434b6f25 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aeeb9a9c-292c-468a-8d2b-ef677e319ea7", "name": "network-vif-deleted", "tag": "4249289a-c974-46ed-9c60-5d0841190f97"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:26.400051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-cea18f36-9284-4de5-8768-89d56162cbe0 req-b6fee393-21b2-4e53-9077-4281434b6f25 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:26.400485 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-cea18f36-9284-4de5-8768-89d56162cbe0 req-b6fee393-21b2-4e53-9077-4281434b6f25 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:26.411001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-cea18f36-9284-4de5-8768-89d56162cbe0 req-b6fee393-21b2-4e53-9077-4281434b6f25 service nova] Creating event network-vif-deleted:4249289a-c974-46ed-9c60-5d0841190f97 for instance aeeb9a9c-292c-468a-8d2b-ef677e319ea7 on hv-cinder-maste Jun 10 13:08:26.415967 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-cea18f36-9284-4de5-8768-89d56162cbe0 req-b6fee393-21b2-4e53-9077-4281434b6f25 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028567 Jun 10 13:08:26.416514 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 242/503] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:08:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:08:26.457413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1cd3d780-05f4-4be3-87b0-67ec13b1092d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:26.458024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1cd3d780-05f4-4be3-87b0-67ec13b1092d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:26.464102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1cd3d780-05f4-4be3-87b0-67ec13b1092d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:26.464548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1cd3d780-05f4-4be3-87b0-67ec13b1092d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:26.465335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1cd3d780-05f4-4be3-87b0-67ec13b1092d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:26.465767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1cd3d780-05f4-4be3-87b0-67ec13b1092d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:26.591250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1cd3d780-05f4-4be3-87b0-67ec13b1092d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:26.598668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1cd3d780-05f4-4be3-87b0-67ec13b1092d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.145087 Jun 10 13:08:26.599237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 262/504] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:26 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:26.860649 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-35158299-32df-4537-bf10-3a14c2900530 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:26.861680 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-35158299-32df-4537-bf10-3a14c2900530 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:26.867934 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-35158299-32df-4537-bf10-3a14c2900530 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:26.868383 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-35158299-32df-4537-bf10-3a14c2900530 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:26.869184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-35158299-32df-4537-bf10-3a14c2900530 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:26.869618 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-35158299-32df-4537-bf10-3a14c2900530 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:26.918295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e5390187-be5d-4acb-ad1d-ddd4066c142b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:26.918994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e5390187-be5d-4acb-ad1d-ddd4066c142b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:26.926209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e5390187-be5d-4acb-ad1d-ddd4066c142b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:26.926740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e5390187-be5d-4acb-ad1d-ddd4066c142b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:26.927565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e5390187-be5d-4acb-ad1d-ddd4066c142b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:26.928057 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e5390187-be5d-4acb-ad1d-ddd4066c142b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:26.958252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e5390187-be5d-4acb-ad1d-ddd4066c142b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:26.958695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e5390187-be5d-4acb-ad1d-ddd4066c142b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:26.959136 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-e5390187-be5d-4acb-ad1d-ddd4066c142b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lazy-loading 'fault' on Instance uuid aeeb9a9c-292c-468a-8d2b-ef677e319ea7 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:08:27.000138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e5390187-be5d-4acb-ad1d-ddd4066c142b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:27.007665 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-35158299-32df-4537-bf10-3a14c2900530 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:27.009046 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e5390187-be5d-4acb-ad1d-ddd4066c142b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1309 microversion: 2.1 time: 0.094197 Jun 10 13:08:27.010078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 263/505] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:26 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1309 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:27.015392 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-35158299-32df-4537-bf10-3a14c2900530 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1509 microversion: 2.1 time: 0.159600 Jun 10 13:08:27.015946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 243/506] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:26 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1509 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:27.119207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-46d426ae-f170-41b5-921c-6eddb0f9ea5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:27.119724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-46d426ae-f170-41b5-921c-6eddb0f9ea5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:27.125779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-46d426ae-f170-41b5-921c-6eddb0f9ea5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:27.126224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-46d426ae-f170-41b5-921c-6eddb0f9ea5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:27.126976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-46d426ae-f170-41b5-921c-6eddb0f9ea5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:27.127409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-46d426ae-f170-41b5-921c-6eddb0f9ea5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:27.175596 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b1ae6a72-07e3-4c0e-bc30-5d0dcef99501 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:27.176201 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b1ae6a72-07e3-4c0e-bc30-5d0dcef99501 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:27.178164 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:27.183035 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b1ae6a72-07e3-4c0e-bc30-5d0dcef99501 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:27.183482 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b1ae6a72-07e3-4c0e-bc30-5d0dcef99501 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:27.184239 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b1ae6a72-07e3-4c0e-bc30-5d0dcef99501 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:27.184671 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b1ae6a72-07e3-4c0e-bc30-5d0dcef99501 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:27.261586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-46d426ae-f170-41b5-921c-6eddb0f9ea5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:27.269265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-46d426ae-f170-41b5-921c-6eddb0f9ea5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.153909 Jun 10 13:08:27.269927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 264/507] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:27 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:27.274962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a8b5925d-1325-4cde-bd13-cda3fb4de401 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:27.275627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a8b5925d-1325-4cde-bd13-cda3fb4de401 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:27.281771 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8b5925d-1325-4cde-bd13-cda3fb4de401 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:27.282926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8b5925d-1325-4cde-bd13-cda3fb4de401 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:27.283769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8b5925d-1325-4cde-bd13-cda3fb4de401 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:27.284210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8b5925d-1325-4cde-bd13-cda3fb4de401 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:27.288338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:27.316335 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b1ae6a72-07e3-4c0e-bc30-5d0dcef99501 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:27.316668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8b5925d-1325-4cde-bd13-cda3fb4de401 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:27.317149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8b5925d-1325-4cde-bd13-cda3fb4de401 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:27.317597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-a8b5925d-1325-4cde-bd13-cda3fb4de401 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lazy-loading 'fault' on Instance uuid 1d2dd021-a498-436e-8f04-13e3b612d24c {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:08:27.325027 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b1ae6a72-07e3-4c0e-bc30-5d0dcef99501 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:27.326523 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b1ae6a72-07e3-4c0e-bc30-5d0dcef99501 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:27.327690 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b1ae6a72-07e3-4c0e-bc30-5d0dcef99501 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.156115 Jun 10 13:08:27.328258 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 244/508] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:27 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:27.362022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a8b5925d-1325-4cde-bd13-cda3fb4de401 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '24343309fb924981a37899f56f15b8ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fdf6af645ba4a02a6a7278eb1fdff0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:27.369979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a8b5925d-1325-4cde-bd13-cda3fb4de401 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 200 len: 1303 microversion: 2.1 time: 0.098846 Jun 10 13:08:27.370487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 265/509] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:27 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 1303 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:27.615843 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6e104c2f-b5fc-48b9-bd0d-90775f0cc3e0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:27.616477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6e104c2f-b5fc-48b9-bd0d-90775f0cc3e0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:27.623059 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e104c2f-b5fc-48b9-bd0d-90775f0cc3e0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:27.623453 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e104c2f-b5fc-48b9-bd0d-90775f0cc3e0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:27.624210 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e104c2f-b5fc-48b9-bd0d-90775f0cc3e0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:27.624640 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e104c2f-b5fc-48b9-bd0d-90775f0cc3e0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:27.754622 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6e104c2f-b5fc-48b9-bd0d-90775f0cc3e0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:27.762043 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6e104c2f-b5fc-48b9-bd0d-90775f0cc3e0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.150366 Jun 10 13:08:27.762599 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 245/510] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:27 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:28.029666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d8f94c3d-5a59-458b-8e65-b2aa6409765b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:28.030285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d8f94c3d-5a59-458b-8e65-b2aa6409765b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:28.031214 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4b9d4d38-687b-41ec-8bbb-93346938b663 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:28.031824 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4b9d4d38-687b-41ec-8bbb-93346938b663 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:28.037973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d8f94c3d-5a59-458b-8e65-b2aa6409765b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:28.038345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d8f94c3d-5a59-458b-8e65-b2aa6409765b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:28.038462 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b9d4d38-687b-41ec-8bbb-93346938b663 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:28.038816 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b9d4d38-687b-41ec-8bbb-93346938b663 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:28.039120 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d8f94c3d-5a59-458b-8e65-b2aa6409765b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:28.039557 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d8f94c3d-5a59-458b-8e65-b2aa6409765b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:28.039652 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b9d4d38-687b-41ec-8bbb-93346938b663 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:28.039986 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b9d4d38-687b-41ec-8bbb-93346938b663 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:28.071014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d8f94c3d-5a59-458b-8e65-b2aa6409765b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:28.071469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d8f94c3d-5a59-458b-8e65-b2aa6409765b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:28.071974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-d8f94c3d-5a59-458b-8e65-b2aa6409765b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lazy-loading 'fault' on Instance uuid aeeb9a9c-292c-468a-8d2b-ef677e319ea7 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:08:28.114655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d8f94c3d-5a59-458b-8e65-b2aa6409765b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:28.124204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d8f94c3d-5a59-458b-8e65-b2aa6409765b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1309 microversion: 2.1 time: 0.097572 Jun 10 13:08:28.124928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 266/511] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:28 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1309 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:28.173604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4b9d4d38-687b-41ec-8bbb-93346938b663 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:28.181465 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4b9d4d38-687b-41ec-8bbb-93346938b663 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1509 microversion: 2.1 time: 0.154085 Jun 10 13:08:28.182044 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 246/512] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:28 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1509 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:28.286950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b8e217bf-ced8-417e-9353-8c90bbe832fc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:28.287496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b8e217bf-ced8-417e-9353-8c90bbe832fc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:28.293659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8e217bf-ced8-417e-9353-8c90bbe832fc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:28.294110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8e217bf-ced8-417e-9353-8c90bbe832fc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:28.294917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8e217bf-ced8-417e-9353-8c90bbe832fc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:28.295410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8e217bf-ced8-417e-9353-8c90bbe832fc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:28.346786 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5efd01a4-28ce-4634-84b1-04c12c43dac8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:28.347480 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5efd01a4-28ce-4634-84b1-04c12c43dac8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:28.353636 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5efd01a4-28ce-4634-84b1-04c12c43dac8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:28.354047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5efd01a4-28ce-4634-84b1-04c12c43dac8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:28.354813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5efd01a4-28ce-4634-84b1-04c12c43dac8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:28.355243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5efd01a4-28ce-4634-84b1-04c12c43dac8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:28.432127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b8e217bf-ced8-417e-9353-8c90bbe832fc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:28.440563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b8e217bf-ced8-417e-9353-8c90bbe832fc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.158271 Jun 10 13:08:28.441182 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 267/513] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:28 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:28.445800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5754c72c-b20b-49fb-a024-35b703e26f10 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:28.446396 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5754c72c-b20b-49fb-a024-35b703e26f10 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] [instance: 1d2dd021-a498-436e-8f04-13e3b612d24c] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:28.452147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5754c72c-b20b-49fb-a024-35b703e26f10 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:28.452599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5754c72c-b20b-49fb-a024-35b703e26f10 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:28.453417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5754c72c-b20b-49fb-a024-35b703e26f10 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:28.456066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5754c72c-b20b-49fb-a024-35b703e26f10 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:28.467797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-5754c72c-b20b-49fb-a024-35b703e26f10 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] HTTP exception thrown: Instance 1d2dd021-a498-436e-8f04-13e3b612d24c could not be found. Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5754c72c-b20b-49fb-a024-35b703e26f10 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Request method failure captured: Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c HTTP/1.1 Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0H689WBWNCTm_Kgo3HOHebBiL8zdvrAVTwjaBaT7O7WxeG7Ru0F6ZlnGiauVQNyjr5U3uKPwYpJucZR8wjbqxuJJNUcu4F7JleSQU0vA2UIHtWi1ZlyReVm_U2_mNKXVDR8kx0ZvsbFDyXEyOd0fTBeIkt6KVXqnZug71yGsoqBV10 Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 0fdf6af645ba4a02a6a7278eb1fdff0d Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesActionsTest-1522136130 Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:08:28.469494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:08:28.471063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/0fdf6af645ba4a02a6a7278eb1fdff0d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0fdf6af645ba4a02a6a7278eb1fdff0d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_0fdf6af645ba4a02a6a7278eb1fdff0d", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0fdf6af645ba4a02a6a7278eb1fdff0d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:08:28.471063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesActionsTest-1522136130 Jun 10 13:08:28.471063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 0fdf6af645ba4a02a6a7278eb1fdff0d Jun 10 13:08:28.471063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesActionsTest-1522136130 Jun 10 13:08:28.471063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesActionsTest-1522136130-project Jun 10 13:08:28.471063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:08:28.471063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:08:28.471063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 24343309fb924981a37899f56f15b8ba Jun 10 13:08:28.471063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesActionsTest-1522136130-project Jun 10 13:08:28.471063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:08:28.471063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 1d2dd021-a498-436e-8f04-13e3b612d24c could not be found. Jun 10 13:08:28.471063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '1d2dd021-a498-436e-8f04-13e3b612d24c'} Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1d2dd021-a498-436e-8f04-13e3b612d24c could not be found. Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:08:28.471932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1d2dd021-a498-436e-8f04-13e3b612d24c could not be found. Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1d2dd021-a498-436e-8f04-13e3b612d24c could not be found. Jun 10 13:08:28.473362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:08:28.474678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5754c72c-b20b-49fb-a024-35b703e26f10 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] Returning 404 to user: Instance 1d2dd021-a498-436e-8f04-13e3b612d24c could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:08:28.474678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5754c72c-b20b-49fb-a024-35b703e26f10 tempest-VolumesActionsTest-1522136130 tempest-VolumesActionsTest-1522136130-project] 10.222.0.25 "GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c" status: 404 len: 111 microversion: 2.1 time: 0.028768 Jun 10 13:08:28.474678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 268/514] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:28 2022] GET /compute/v2.1/servers/1d2dd021-a498-436e-8f04-13e3b612d24c => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:08:28.505324 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5efd01a4-28ce-4634-84b1-04c12c43dac8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:28.514102 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5efd01a4-28ce-4634-84b1-04c12c43dac8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:28.515641 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5efd01a4-28ce-4634-84b1-04c12c43dac8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:28.525548 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5efd01a4-28ce-4634-84b1-04c12c43dac8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.174963 Jun 10 13:08:28.525548 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 247/515] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:28 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:28.780826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5e1fd6e6-d323-4a18-81fb-6c3df69a5c4c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:28.782072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5e1fd6e6-d323-4a18-81fb-6c3df69a5c4c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:28.789486 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e1fd6e6-d323-4a18-81fb-6c3df69a5c4c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:28.790026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e1fd6e6-d323-4a18-81fb-6c3df69a5c4c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:28.790953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e1fd6e6-d323-4a18-81fb-6c3df69a5c4c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:28.791511 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e1fd6e6-d323-4a18-81fb-6c3df69a5c4c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:28.925168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5e1fd6e6-d323-4a18-81fb-6c3df69a5c4c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:28.932866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5e1fd6e6-d323-4a18-81fb-6c3df69a5c4c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.156851 Jun 10 13:08:28.933421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 269/516] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:28 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:29.141422 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2db227a4-0ad5-4efa-bdca-fe7537a093c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:29.142117 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2db227a4-0ad5-4efa-bdca-fe7537a093c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:29.148539 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2db227a4-0ad5-4efa-bdca-fe7537a093c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:29.149084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2db227a4-0ad5-4efa-bdca-fe7537a093c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:29.149851 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2db227a4-0ad5-4efa-bdca-fe7537a093c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:29.150283 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2db227a4-0ad5-4efa-bdca-fe7537a093c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:29.181655 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2db227a4-0ad5-4efa-bdca-fe7537a093c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:29.182019 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2db227a4-0ad5-4efa-bdca-fe7537a093c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:29.182470 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-2db227a4-0ad5-4efa-bdca-fe7537a093c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lazy-loading 'fault' on Instance uuid aeeb9a9c-292c-468a-8d2b-ef677e319ea7 {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:08:29.198574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7d35d49c-a168-4f80-ae20-9409e1c7dd05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:29.199282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7d35d49c-a168-4f80-ae20-9409e1c7dd05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:29.206000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d35d49c-a168-4f80-ae20-9409e1c7dd05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:29.206509 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d35d49c-a168-4f80-ae20-9409e1c7dd05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:29.207267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d35d49c-a168-4f80-ae20-9409e1c7dd05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:29.210102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d35d49c-a168-4f80-ae20-9409e1c7dd05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:29.222887 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2db227a4-0ad5-4efa-bdca-fe7537a093c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:29.230384 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2db227a4-0ad5-4efa-bdca-fe7537a093c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 200 len: 1309 microversion: 2.1 time: 0.092846 Jun 10 13:08:29.230948 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 248/517] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:29 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 1309 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:29.347173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7d35d49c-a168-4f80-ae20-9409e1c7dd05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:29.354836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7d35d49c-a168-4f80-ae20-9409e1c7dd05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1515 microversion: 2.1 time: 0.160180 Jun 10 13:08:29.355466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 270/518] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:29 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1515 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:29.461556 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1d5f3156-832f-4a29-9259-6217dc0499e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:29.462164 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1d5f3156-832f-4a29-9259-6217dc0499e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:29.470885 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1d5f3156-832f-4a29-9259-6217dc0499e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:29.471061 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1d5f3156-832f-4a29-9259-6217dc0499e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:29.471690 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1d5f3156-832f-4a29-9259-6217dc0499e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:29.472150 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1d5f3156-832f-4a29-9259-6217dc0499e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:29.539277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-790ac644-379c-4797-9eb6-44e12c5cf42e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:29.541087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-790ac644-379c-4797-9eb6-44e12c5cf42e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:29.547738 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-790ac644-379c-4797-9eb6-44e12c5cf42e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:29.548405 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-790ac644-379c-4797-9eb6-44e12c5cf42e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:29.549206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-790ac644-379c-4797-9eb6-44e12c5cf42e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:29.549638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-790ac644-379c-4797-9eb6-44e12c5cf42e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:29.647573 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1d5f3156-832f-4a29-9259-6217dc0499e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:29.657733 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1d5f3156-832f-4a29-9259-6217dc0499e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.199451 Jun 10 13:08:29.658698 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 249/519] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:29 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:29.708146 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-790ac644-379c-4797-9eb6-44e12c5cf42e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:29.716977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-790ac644-379c-4797-9eb6-44e12c5cf42e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:29.718615 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-790ac644-379c-4797-9eb6-44e12c5cf42e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:29.720441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-790ac644-379c-4797-9eb6-44e12c5cf42e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.188211 Jun 10 13:08:29.721108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 271/520] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:29 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:29.950507 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e1ead6cf-ed1f-4bf7-9e69-22d57009127d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:29.951141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e1ead6cf-ed1f-4bf7-9e69-22d57009127d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:29.961821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e1ead6cf-ed1f-4bf7-9e69-22d57009127d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:29.961821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e1ead6cf-ed1f-4bf7-9e69-22d57009127d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:29.961821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e1ead6cf-ed1f-4bf7-9e69-22d57009127d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:29.961821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e1ead6cf-ed1f-4bf7-9e69-22d57009127d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:30.088983 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e1ead6cf-ed1f-4bf7-9e69-22d57009127d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:30.096559 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e1ead6cf-ed1f-4bf7-9e69-22d57009127d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.149827 Jun 10 13:08:30.097163 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 250/521] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:29 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:30.247490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a8045741-5ac8-401b-82fa-cb059bc9b20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:30.247999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a8045741-5ac8-401b-82fa-cb059bc9b20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: aeeb9a9c-292c-468a-8d2b-ef677e319ea7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:30.255205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8045741-5ac8-401b-82fa-cb059bc9b20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:30.255659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8045741-5ac8-401b-82fa-cb059bc9b20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:30.256395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8045741-5ac8-401b-82fa-cb059bc9b20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:30.256849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8045741-5ac8-401b-82fa-cb059bc9b20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:30.269915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-a8045741-5ac8-401b-82fa-cb059bc9b20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] HTTP exception thrown: Instance aeeb9a9c-292c-468a-8d2b-ef677e319ea7 could not be found. Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a8045741-5ac8-401b-82fa-cb059bc9b20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Request method failure captured: Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 HTTP/1.1 Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0HlyICQX9l5KFYyUsFzMtUgTAzPRfxQzNN7dM9kV13g0iWQTn9QMENYtfZqE20ETOnCUUxvj1yudvSxd1bFTvMSIvc_eVAfoGvb2Phk1I--eUGA8-ePhD-Rp01rKEhzSDpYm-ppUB1HJXel5mUDpZwxED5NTdU7SM0nV7mlEGF1wbo Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: ff2616337ae440a89047fd9a50218387 Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:08:30.271409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:08:30.277468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ff2616337ae440a89047fd9a50218387"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ff2616337ae440a89047fd9a50218387"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ff2616337ae440a89047fd9a50218387", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ff2616337ae440a89047fd9a50218387"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:08:30.277468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:08:30.277468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: ff2616337ae440a89047fd9a50218387 Jun 10 13:08:30.277468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:08:30.277468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesSnapshotTestJSON-1698707508-project Jun 10 13:08:30.277468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:08:30.277468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:08:30.277468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 849e36f8653d4d1f9a7fb1df8fb913a9 Jun 10 13:08:30.277468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesSnapshotTestJSON-1698707508-project Jun 10 13:08:30.277468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:08:30.277468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance aeeb9a9c-292c-468a-8d2b-ef677e319ea7 could not be found. Jun 10 13:08:30.277468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'aeeb9a9c-292c-468a-8d2b-ef677e319ea7'} Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aeeb9a9c-292c-468a-8d2b-ef677e319ea7 could not be found. Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:08:30.278368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aeeb9a9c-292c-468a-8d2b-ef677e319ea7 could not be found. Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aeeb9a9c-292c-468a-8d2b-ef677e319ea7 could not be found. Jun 10 13:08:30.279748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:08:30.281032 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a8045741-5ac8-401b-82fa-cb059bc9b20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Returning 404 to user: Instance aeeb9a9c-292c-468a-8d2b-ef677e319ea7 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:08:30.281032 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a8045741-5ac8-401b-82fa-cb059bc9b20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7" status: 404 len: 111 microversion: 2.1 time: 0.029356 Jun 10 13:08:30.281032 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 272/522] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:30 2022] GET /compute/v2.1/servers/aeeb9a9c-292c-468a-8d2b-ef677e319ea7 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:08:30.675042 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8eb5aa0f-5832-4253-8607-83e0fbf54cd9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:30.675776 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8eb5aa0f-5832-4253-8607-83e0fbf54cd9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:30.682320 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8eb5aa0f-5832-4253-8607-83e0fbf54cd9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:30.682764 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8eb5aa0f-5832-4253-8607-83e0fbf54cd9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:30.683524 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8eb5aa0f-5832-4253-8607-83e0fbf54cd9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:30.683951 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8eb5aa0f-5832-4253-8607-83e0fbf54cd9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:30.739971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-24dfaff7-3436-4342-bb9a-97a8a50ab892 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:30.740585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-24dfaff7-3436-4342-bb9a-97a8a50ab892 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:30.747458 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-24dfaff7-3436-4342-bb9a-97a8a50ab892 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:30.747907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-24dfaff7-3436-4342-bb9a-97a8a50ab892 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:30.748669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-24dfaff7-3436-4342-bb9a-97a8a50ab892 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:30.749251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-24dfaff7-3436-4342-bb9a-97a8a50ab892 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:30.811084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8eb5aa0f-5832-4253-8607-83e0fbf54cd9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:30.818577 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8eb5aa0f-5832-4253-8607-83e0fbf54cd9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.148087 Jun 10 13:08:30.819147 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 251/523] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:30 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:30.875578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-24dfaff7-3436-4342-bb9a-97a8a50ab892 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:30.884601 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-24dfaff7-3436-4342-bb9a-97a8a50ab892 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:30.886123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-24dfaff7-3436-4342-bb9a-97a8a50ab892 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:30.887274 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-24dfaff7-3436-4342-bb9a-97a8a50ab892 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.152017 Jun 10 13:08:30.887803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 273/524] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:30 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:31.113958 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f6415440-1a56-4255-bc8c-afccee0a5d53 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:31.114494 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f6415440-1a56-4255-bc8c-afccee0a5d53 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:31.120544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6415440-1a56-4255-bc8c-afccee0a5d53 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:31.121035 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6415440-1a56-4255-bc8c-afccee0a5d53 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:31.121802 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6415440-1a56-4255-bc8c-afccee0a5d53 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:31.122235 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6415440-1a56-4255-bc8c-afccee0a5d53 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:31.248966 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f6415440-1a56-4255-bc8c-afccee0a5d53 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:31.256868 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f6415440-1a56-4255-bc8c-afccee0a5d53 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.147053 Jun 10 13:08:31.257415 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 252/525] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:31 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:31.835144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5e63987c-688f-443a-bcc2-c068f7752fc5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:31.835741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5e63987c-688f-443a-bcc2-c068f7752fc5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:31.841826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e63987c-688f-443a-bcc2-c068f7752fc5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:31.842269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e63987c-688f-443a-bcc2-c068f7752fc5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:31.843016 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e63987c-688f-443a-bcc2-c068f7752fc5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:31.843445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e63987c-688f-443a-bcc2-c068f7752fc5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:31.905373 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3e9b735f-8ac4-40ac-a496-4f23ae2d83ad tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:31.905969 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3e9b735f-8ac4-40ac-a496-4f23ae2d83ad tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:31.912356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3e9b735f-8ac4-40ac-a496-4f23ae2d83ad tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:31.912814 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3e9b735f-8ac4-40ac-a496-4f23ae2d83ad tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:31.913572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3e9b735f-8ac4-40ac-a496-4f23ae2d83ad tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:31.913993 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3e9b735f-8ac4-40ac-a496-4f23ae2d83ad tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:31.973096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5e63987c-688f-443a-bcc2-c068f7752fc5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:31.980793 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5e63987c-688f-443a-bcc2-c068f7752fc5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.149658 Jun 10 13:08:31.981473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 274/526] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:31 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:32.041543 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3e9b735f-8ac4-40ac-a496-4f23ae2d83ad tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:32.050610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3e9b735f-8ac4-40ac-a496-4f23ae2d83ad tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:32.052092 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3e9b735f-8ac4-40ac-a496-4f23ae2d83ad tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:32.053271 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3e9b735f-8ac4-40ac-a496-4f23ae2d83ad tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.151695 Jun 10 13:08:32.053827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 253/527] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:31 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:32.273867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-233e6371-0381-4f66-aeaf-4f1eacbe260f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:32.274468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-233e6371-0381-4f66-aeaf-4f1eacbe260f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:32.280512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-233e6371-0381-4f66-aeaf-4f1eacbe260f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:32.280993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-233e6371-0381-4f66-aeaf-4f1eacbe260f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:32.281745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-233e6371-0381-4f66-aeaf-4f1eacbe260f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:32.282168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-233e6371-0381-4f66-aeaf-4f1eacbe260f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:32.409009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-233e6371-0381-4f66-aeaf-4f1eacbe260f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:32.416258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-233e6371-0381-4f66-aeaf-4f1eacbe260f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.146235 Jun 10 13:08:32.416832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 275/528] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:32 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:32.998461 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ae6854cc-95ce-4b02-8da1-454dfa07bdd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:32.999057 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ae6854cc-95ce-4b02-8da1-454dfa07bdd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:33.006679 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ae6854cc-95ce-4b02-8da1-454dfa07bdd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:33.007119 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ae6854cc-95ce-4b02-8da1-454dfa07bdd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:33.007870 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ae6854cc-95ce-4b02-8da1-454dfa07bdd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:33.008293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ae6854cc-95ce-4b02-8da1-454dfa07bdd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:33.071371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-80c33751-42d0-4600-b7ce-594254d7da01 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:33.072089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-80c33751-42d0-4600-b7ce-594254d7da01 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:33.079047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80c33751-42d0-4600-b7ce-594254d7da01 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:33.079588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80c33751-42d0-4600-b7ce-594254d7da01 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:33.080522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80c33751-42d0-4600-b7ce-594254d7da01 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:33.080971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80c33751-42d0-4600-b7ce-594254d7da01 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:33.154092 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ae6854cc-95ce-4b02-8da1-454dfa07bdd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:33.162531 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ae6854cc-95ce-4b02-8da1-454dfa07bdd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.168406 Jun 10 13:08:33.163112 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 254/529] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:32 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:33.218899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-80c33751-42d0-4600-b7ce-594254d7da01 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:33.228040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-80c33751-42d0-4600-b7ce-594254d7da01 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:33.229561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-80c33751-42d0-4600-b7ce-594254d7da01 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:33.230710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-80c33751-42d0-4600-b7ce-594254d7da01 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.163414 Jun 10 13:08:33.231265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 276/530] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:33 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:33.433106 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-80bb37b9-f9c8-4272-bfed-eec40a34841c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:33.433638 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-80bb37b9-f9c8-4272-bfed-eec40a34841c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:33.439766 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-80bb37b9-f9c8-4272-bfed-eec40a34841c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:33.440213 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-80bb37b9-f9c8-4272-bfed-eec40a34841c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:33.441202 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-80bb37b9-f9c8-4272-bfed-eec40a34841c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:33.441640 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-80bb37b9-f9c8-4272-bfed-eec40a34841c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:33.566675 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-80bb37b9-f9c8-4272-bfed-eec40a34841c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:33.574203 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-80bb37b9-f9c8-4272-bfed-eec40a34841c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.145081 Jun 10 13:08:33.574763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 255/531] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:33 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:34.166725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:08:34.167237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:08:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:08:34.179454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8a36d08a-db0d-475f-a349-beae8f7fbe3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:34.180088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8a36d08a-db0d-475f-a349-beae8f7fbe3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:34.186177 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-40d29354-38df-4566-b13c-f4e3e5ddb49f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Action: 'create', calling method: .version_select at 0x7f93052bb9d0>, body: {"keypair": {"name": "tempest-keypair-80950017", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGwRcpbUSDnIgTnNia/UunpIaj3V+MrFMKidlNy5fWa8NaB7+eOvW1GK/0ztV5HG9jw+ZeqiYsARC6YgXA1GVUVTB6dONotbDsM+SWbxO+vSpPDahCvOPU9ML32IjPWAtw=="}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:34.187138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a36d08a-db0d-475f-a349-beae8f7fbe3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:34.187585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a36d08a-db0d-475f-a349-beae8f7fbe3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:34.188343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a36d08a-db0d-475f-a349-beae8f7fbe3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:34.188836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a36d08a-db0d-475f-a349-beae8f7fbe3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:34.200834 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-40d29354-38df-4566-b13c-f4e3e5ddb49f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Getting quotas for project 03423e3482614374afa8c929bd931262. Resources: {'key_pairs'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:08:34.204834 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-40d29354-38df-4566-b13c-f4e3e5ddb49f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Getting quotas for user 25f73d0bd4e64057a922d11be46eb04c and project 03423e3482614374afa8c929bd931262. Resources: {'key_pairs'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:08:34.293418 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-40d29354-38df-4566-b13c-f4e3e5ddb49f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.110874 Jun 10 13:08:34.294369 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 256/532] 10.222.0.25 () {60 vars in 1158 bytes} [Fri Jun 10 13:08:34 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:08:34.299851 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4dd4acad-b9be-4987-aee7-990daca8f8c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:34.301154 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4dd4acad-b9be-4987-aee7-990daca8f8c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:34.309321 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4dd4acad-b9be-4987-aee7-990daca8f8c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:34.309767 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4dd4acad-b9be-4987-aee7-990daca8f8c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:34.310710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4dd4acad-b9be-4987-aee7-990daca8f8c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:34.310962 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4dd4acad-b9be-4987-aee7-990daca8f8c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:34.328484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8a36d08a-db0d-475f-a349-beae8f7fbe3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:34.337708 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8a36d08a-db0d-475f-a349-beae8f7fbe3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.161900 Jun 10 13:08:34.338723 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 277/533] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:34 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:34.491242 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4dd4acad-b9be-4987-aee7-990daca8f8c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:34.499595 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4dd4acad-b9be-4987-aee7-990daca8f8c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:34.501172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4dd4acad-b9be-4987-aee7-990daca8f8c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:34.502462 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4dd4acad-b9be-4987-aee7-990daca8f8c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.206854 Jun 10 13:08:34.503017 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 257/534] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:34 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:08:34.601273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-40fb3801-ba5f-4318-a989-fb10824a14c0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:34.601886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-40fb3801-ba5f-4318-a989-fb10824a14c0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:34.607876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-40fb3801-ba5f-4318-a989-fb10824a14c0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:34.608331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-40fb3801-ba5f-4318-a989-fb10824a14c0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:34.609129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-40fb3801-ba5f-4318-a989-fb10824a14c0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:34.609564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-40fb3801-ba5f-4318-a989-fb10824a14c0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:34.740232 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-40fb3801-ba5f-4318-a989-fb10824a14c0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:34.748576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-40fb3801-ba5f-4318-a989-fb10824a14c0 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.151335 Jun 10 13:08:34.758218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 278/535] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:34 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:35.355074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-af2b488d-77ac-4864-8e2e-a9c203b28f14 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:35.355678 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-af2b488d-77ac-4864-8e2e-a9c203b28f14 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:35.362835 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-af2b488d-77ac-4864-8e2e-a9c203b28f14 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:35.363337 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-af2b488d-77ac-4864-8e2e-a9c203b28f14 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:35.364100 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-af2b488d-77ac-4864-8e2e-a9c203b28f14 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:35.364524 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-af2b488d-77ac-4864-8e2e-a9c203b28f14 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:35.520824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e112524c-31a1-48b4-adc7-f25a1636058e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:35.521440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e112524c-31a1-48b4-adc7-f25a1636058e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:35.527746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e112524c-31a1-48b4-adc7-f25a1636058e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:35.529044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e112524c-31a1-48b4-adc7-f25a1636058e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:35.529044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e112524c-31a1-48b4-adc7-f25a1636058e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:35.529482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e112524c-31a1-48b4-adc7-f25a1636058e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:35.584883 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-af2b488d-77ac-4864-8e2e-a9c203b28f14 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:35.592878 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-af2b488d-77ac-4864-8e2e-a9c203b28f14 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.241560 Jun 10 13:08:35.593671 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 258/536] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:35 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:35.670540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e112524c-31a1-48b4-adc7-f25a1636058e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:35.679840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e112524c-31a1-48b4-adc7-f25a1636058e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:35.681385 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e112524c-31a1-48b4-adc7-f25a1636058e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:35.682635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e112524c-31a1-48b4-adc7-f25a1636058e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.165822 Jun 10 13:08:35.683205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 279/537] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:35 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:35.765030 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-19478952-065c-4d27-9d02-9d3ea4b15caa tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:35.765638 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-19478952-065c-4d27-9d02-9d3ea4b15caa tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:35.772339 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-19478952-065c-4d27-9d02-9d3ea4b15caa tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:35.772502 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-19478952-065c-4d27-9d02-9d3ea4b15caa tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:35.773288 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-19478952-065c-4d27-9d02-9d3ea4b15caa tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:35.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-19478952-065c-4d27-9d02-9d3ea4b15caa tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:35.777834 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:36.002403 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-19478952-065c-4d27-9d02-9d3ea4b15caa tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:36.009956 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-19478952-065c-4d27-9d02-9d3ea4b15caa tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.248755 Jun 10 13:08:36.010590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 259/538] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:35 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:36.609826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2d370a56-fe6b-4574-b4e6-8b97620ea86b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:36.610435 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2d370a56-fe6b-4574-b4e6-8b97620ea86b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:36.617251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d370a56-fe6b-4574-b4e6-8b97620ea86b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:36.617704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d370a56-fe6b-4574-b4e6-8b97620ea86b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:36.619825 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d370a56-fe6b-4574-b4e6-8b97620ea86b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:36.619825 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d370a56-fe6b-4574-b4e6-8b97620ea86b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:36.658353 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1052164214", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a8e4f8d9-a40b-43e6-a232-daac66b17f5f"}], "security_groups": [{"name": "tempest-securitygroup--1308412978"}], "key_name": "tempest-keypair-80950017", "user_data": "}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:36.746383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2d370a56-fe6b-4574-b4e6-8b97620ea86b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:36.754341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2d370a56-fe6b-4574-b4e6-8b97620ea86b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.148481 Jun 10 13:08:36.763586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 280/539] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:36 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:36.763586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-861424e2-92d6-49a6-ad35-b8b63e20d4ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:36.763586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-861424e2-92d6-49a6-ad35-b8b63e20d4ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:36.766391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-861424e2-92d6-49a6-ad35-b8b63e20d4ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:36.766839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-861424e2-92d6-49a6-ad35-b8b63e20d4ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:36.768227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-861424e2-92d6-49a6-ad35-b8b63e20d4ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:36.768655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-861424e2-92d6-49a6-ad35-b8b63e20d4ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:36.774362 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.network.neutron [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] validate_networks() for [('a8e4f8d9-a40b-43e6-a232-daac66b17f5f', None, None, None, None, None)] {{(pid=118093) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:08:36.903889 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-861424e2-92d6-49a6-ad35-b8b63e20d4ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:36.911949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-861424e2-92d6-49a6-ad35-b8b63e20d4ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:36.913572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-861424e2-92d6-49a6-ad35-b8b63e20d4ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:36.914734 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-861424e2-92d6-49a6-ad35-b8b63e20d4ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.158828 Jun 10 13:08:36.915498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 281/540] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:36 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:08:37.035407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b5a94013-d41e-4dda-9ceb-cd28373fa671 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:37.036665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b5a94013-d41e-4dda-9ceb-cd28373fa671 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:37.044217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5a94013-d41e-4dda-9ceb-cd28373fa671 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:37.044729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5a94013-d41e-4dda-9ceb-cd28373fa671 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:37.045982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5a94013-d41e-4dda-9ceb-cd28373fa671 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:37.046142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5a94013-d41e-4dda-9ceb-cd28373fa671 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:37.069770 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:37.234133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b5a94013-d41e-4dda-9ceb-cd28373fa671 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:37.246499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b5a94013-d41e-4dda-9ceb-cd28373fa671 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.220413 Jun 10 13:08:37.246499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 282/541] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:37 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:37.476701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Flavor limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:08:37.477623 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Image limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:08:37.480355 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Flavor pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:08:37.480355 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Image pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:08:37.480355 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:08:37.493932 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:37.494888 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:37.495832 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:37.496650 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:37.521961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Getting quotas for project 03423e3482614374afa8c929bd931262. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:08:37.529175 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Getting quotas for user 25f73d0bd4e64057a922d11be46eb04c and project 03423e3482614374afa8c929bd931262. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:08:37.544245 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Going to run 1 instances... {{(pid=118093) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:08:37.548447 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118093) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:08:37.723671 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-13bad534-837f-4ca2-ac91-0c091abdec61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.068644 Jun 10 13:08:37.724315 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 260/542] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:08:36 2022] POST /compute/v2.1/servers => generated 409 bytes in 1071 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jun 10 13:08:37.734563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-aeb99994-b22c-4367-9b8e-b1a34b11d334 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:37.737263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-aeb99994-b22c-4367-9b8e-b1a34b11d334 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:37.771431 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9befdb34-e0c2-4219-81a3-a739936f8a0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:37.772434 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9befdb34-e0c2-4219-81a3-a739936f8a0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:37.781631 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9befdb34-e0c2-4219-81a3-a739936f8a0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:37.782116 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9befdb34-e0c2-4219-81a3-a739936f8a0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:37.783693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9befdb34-e0c2-4219-81a3-a739936f8a0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:37.784627 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9befdb34-e0c2-4219-81a3-a739936f8a0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:37.810498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-aeb99994-b22c-4367-9b8e-b1a34b11d334 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:37.828220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-aeb99994-b22c-4367-9b8e-b1a34b11d334 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1244 microversion: 2.1 time: 0.093005 Jun 10 13:08:37.829479 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 283/543] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:37 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1244 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:37.933920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b8e4d9d5-6b98-4700-b4d7-a0cfb23bacb5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:37.934605 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b8e4d9d5-6b98-4700-b4d7-a0cfb23bacb5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:37.957958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8e4d9d5-6b98-4700-b4d7-a0cfb23bacb5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:37.957958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8e4d9d5-6b98-4700-b4d7-a0cfb23bacb5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:37.959286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8e4d9d5-6b98-4700-b4d7-a0cfb23bacb5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:37.961260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8e4d9d5-6b98-4700-b4d7-a0cfb23bacb5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:38.063395 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9befdb34-e0c2-4219-81a3-a739936f8a0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:38.072981 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9befdb34-e0c2-4219-81a3-a739936f8a0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.306275 Jun 10 13:08:38.073720 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 261/544] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:37 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:38.149463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b8e4d9d5-6b98-4700-b4d7-a0cfb23bacb5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:38.163392 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b8e4d9d5-6b98-4700-b4d7-a0cfb23bacb5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:38.164580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b8e4d9d5-6b98-4700-b4d7-a0cfb23bacb5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:38.165914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b8e4d9d5-6b98-4700-b4d7-a0cfb23bacb5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1807 microversion: 2.20 time: 0.235867 Jun 10 13:08:38.166544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 284/545] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:37 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1807 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:08:38.264118 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-be3ea99c-ba95-4068-9168-eaf85659f77f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:38.264861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-be3ea99c-ba95-4068-9168-eaf85659f77f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:38.271416 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-be3ea99c-ba95-4068-9168-eaf85659f77f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:38.272093 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-be3ea99c-ba95-4068-9168-eaf85659f77f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:38.308391 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-be3ea99c-ba95-4068-9168-eaf85659f77f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:38.308885 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-be3ea99c-ba95-4068-9168-eaf85659f77f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:38.536940 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-be3ea99c-ba95-4068-9168-eaf85659f77f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:38.547058 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-be3ea99c-ba95-4068-9168-eaf85659f77f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.287091 Jun 10 13:08:38.547641 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 262/546] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:38 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:38.847553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ad5f80fd-f683-4fcc-84e6-2ecfd0641cc0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:38.848222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ad5f80fd-f683-4fcc-84e6-2ecfd0641cc0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:38.854003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:38.856084 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad5f80fd-f683-4fcc-84e6-2ecfd0641cc0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:38.856915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad5f80fd-f683-4fcc-84e6-2ecfd0641cc0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:38.857683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad5f80fd-f683-4fcc-84e6-2ecfd0641cc0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:38.858109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad5f80fd-f683-4fcc-84e6-2ecfd0641cc0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:38.966501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ad5f80fd-f683-4fcc-84e6-2ecfd0641cc0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:38.975610 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ad5f80fd-f683-4fcc-84e6-2ecfd0641cc0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1248 microversion: 2.1 time: 0.132619 Jun 10 13:08:38.985362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 285/547] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:38 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1248 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:39.091455 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5a0eae2a-063d-43d7-9f67-995cadef1f68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:39.093449 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5a0eae2a-063d-43d7-9f67-995cadef1f68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:39.102153 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a0eae2a-063d-43d7-9f67-995cadef1f68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:39.102598 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a0eae2a-063d-43d7-9f67-995cadef1f68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:39.103600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a0eae2a-063d-43d7-9f67-995cadef1f68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:39.104467 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a0eae2a-063d-43d7-9f67-995cadef1f68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:39.195574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4df732dc-92a6-41a4-8299-fbaf9a65c65b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:39.200957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4df732dc-92a6-41a4-8299-fbaf9a65c65b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:39.214792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4df732dc-92a6-41a4-8299-fbaf9a65c65b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:39.221470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4df732dc-92a6-41a4-8299-fbaf9a65c65b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:39.221925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4df732dc-92a6-41a4-8299-fbaf9a65c65b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:39.224033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4df732dc-92a6-41a4-8299-fbaf9a65c65b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:39.300786 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5a0eae2a-063d-43d7-9f67-995cadef1f68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:39.316025 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5a0eae2a-063d-43d7-9f67-995cadef1f68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.228003 Jun 10 13:08:39.317750 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 263/548] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:39 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:39.452666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4df732dc-92a6-41a4-8299-fbaf9a65c65b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:39.470652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4df732dc-92a6-41a4-8299-fbaf9a65c65b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:39.472668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4df732dc-92a6-41a4-8299-fbaf9a65c65b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:39.478275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4df732dc-92a6-41a4-8299-fbaf9a65c65b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1794 microversion: 2.20 time: 0.292769 Jun 10 13:08:39.478275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 286/549] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:08:39 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1794 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:08:39.565510 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b62644f1-7576-425a-9638-c6010a9cb97c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:39.566648 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b62644f1-7576-425a-9638-c6010a9cb97c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:39.574503 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b62644f1-7576-425a-9638-c6010a9cb97c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:39.576134 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b62644f1-7576-425a-9638-c6010a9cb97c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:39.576637 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b62644f1-7576-425a-9638-c6010a9cb97c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:39.577304 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b62644f1-7576-425a-9638-c6010a9cb97c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:39.760893 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b62644f1-7576-425a-9638-c6010a9cb97c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:39.772190 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b62644f1-7576-425a-9638-c6010a9cb97c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.210800 Jun 10 13:08:39.773185 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 264/550] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:39 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:39.993179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d533dc84-74c1-46d2-aa73-e12c85017c75 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:39.993803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d533dc84-74c1-46d2-aa73-e12c85017c75 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:40.000953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d533dc84-74c1-46d2-aa73-e12c85017c75 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:40.001698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d533dc84-74c1-46d2-aa73-e12c85017c75 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:40.002763 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d533dc84-74c1-46d2-aa73-e12c85017c75 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:40.003384 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d533dc84-74c1-46d2-aa73-e12c85017c75 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:40.085854 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d533dc84-74c1-46d2-aa73-e12c85017c75 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:40.099409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d533dc84-74c1-46d2-aa73-e12c85017c75 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1304 microversion: 2.1 time: 0.109985 Jun 10 13:08:40.100370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 287/551] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:39 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1304 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:40.336437 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6cb44c18-f0c2-4f16-8f55-42837eaec157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:40.337758 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6cb44c18-f0c2-4f16-8f55-42837eaec157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:40.344282 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6cb44c18-f0c2-4f16-8f55-42837eaec157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:40.344719 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6cb44c18-f0c2-4f16-8f55-42837eaec157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:40.345516 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6cb44c18-f0c2-4f16-8f55-42837eaec157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:40.345944 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6cb44c18-f0c2-4f16-8f55-42837eaec157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:40.511226 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6cb44c18-f0c2-4f16-8f55-42837eaec157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:40.519223 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6cb44c18-f0c2-4f16-8f55-42837eaec157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.186832 Jun 10 13:08:40.519789 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 265/552] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:40 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:40.790022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7ff7d819-de59-42f0-b123-15d686dacbd4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:40.790638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7ff7d819-de59-42f0-b123-15d686dacbd4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:40.797184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ff7d819-de59-42f0-b123-15d686dacbd4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:40.797651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ff7d819-de59-42f0-b123-15d686dacbd4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:40.799030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ff7d819-de59-42f0-b123-15d686dacbd4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:40.799368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ff7d819-de59-42f0-b123-15d686dacbd4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:40.944571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7ff7d819-de59-42f0-b123-15d686dacbd4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:40.952790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7ff7d819-de59-42f0-b123-15d686dacbd4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.166908 Jun 10 13:08:40.953434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 288/553] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:40 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:41.124067 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f460f07d-60c4-4d45-afc4-e20652e6211b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:41.124688 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f460f07d-60c4-4d45-afc4-e20652e6211b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:41.132230 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f460f07d-60c4-4d45-afc4-e20652e6211b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:41.132822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f460f07d-60c4-4d45-afc4-e20652e6211b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:41.133954 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f460f07d-60c4-4d45-afc4-e20652e6211b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:41.134482 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f460f07d-60c4-4d45-afc4-e20652e6211b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:41.203173 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f460f07d-60c4-4d45-afc4-e20652e6211b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:41.213768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f460f07d-60c4-4d45-afc4-e20652e6211b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1314 microversion: 2.1 time: 0.093365 Jun 10 13:08:41.215496 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 266/554] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:41 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1314 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:41.536838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4ed3a99c-6586-446c-9854-8cd5d6399343 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:41.538230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4ed3a99c-6586-446c-9854-8cd5d6399343 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:41.546550 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4ed3a99c-6586-446c-9854-8cd5d6399343 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:41.547327 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4ed3a99c-6586-446c-9854-8cd5d6399343 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:41.548384 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4ed3a99c-6586-446c-9854-8cd5d6399343 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:41.549076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4ed3a99c-6586-446c-9854-8cd5d6399343 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:41.699752 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4ed3a99c-6586-446c-9854-8cd5d6399343 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:41.707330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4ed3a99c-6586-446c-9854-8cd5d6399343 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.174898 Jun 10 13:08:41.708408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 289/555] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:41 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:41.981210 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-bbb2d26f-10c4-4f0e-bf96-d22f509fda74 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:41.982215 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-bbb2d26f-10c4-4f0e-bf96-d22f509fda74 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:41.989962 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bbb2d26f-10c4-4f0e-bf96-d22f509fda74 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:41.990630 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bbb2d26f-10c4-4f0e-bf96-d22f509fda74 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:41.991568 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bbb2d26f-10c4-4f0e-bf96-d22f509fda74 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:41.991995 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bbb2d26f-10c4-4f0e-bf96-d22f509fda74 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:42.244675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-20141e66-a148-4cd4-84d8-2eb6a7b757c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:42.244675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-20141e66-a148-4cd4-84d8-2eb6a7b757c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:42.254234 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-bbb2d26f-10c4-4f0e-bf96-d22f509fda74 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:42.258773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20141e66-a148-4cd4-84d8-2eb6a7b757c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:42.259233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20141e66-a148-4cd4-84d8-2eb6a7b757c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:42.260806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20141e66-a148-4cd4-84d8-2eb6a7b757c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:42.261252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20141e66-a148-4cd4-84d8-2eb6a7b757c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:42.262965 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-bbb2d26f-10c4-4f0e-bf96-d22f509fda74 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.287998 Jun 10 13:08:42.272667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 267/556] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:41 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:42.358468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-20141e66-a148-4cd4-84d8-2eb6a7b757c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:42.367703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-20141e66-a148-4cd4-84d8-2eb6a7b757c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1314 microversion: 2.1 time: 0.139678 Jun 10 13:08:42.368297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 290/557] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:42 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1314 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:42.731393 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-789039cf-862b-4964-9d92-bc2b43ee3879 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:42.732470 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-789039cf-862b-4964-9d92-bc2b43ee3879 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:42.741950 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-789039cf-862b-4964-9d92-bc2b43ee3879 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:42.742391 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-789039cf-862b-4964-9d92-bc2b43ee3879 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:42.743155 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-789039cf-862b-4964-9d92-bc2b43ee3879 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:42.743581 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-789039cf-862b-4964-9d92-bc2b43ee3879 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:42.942167 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-789039cf-862b-4964-9d92-bc2b43ee3879 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:42.950006 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-789039cf-862b-4964-9d92-bc2b43ee3879 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.227802 Jun 10 13:08:42.950570 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 268/558] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:42 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:43.279757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9bff8c48-7a1b-4d96-bf5c-229d82ebaef5 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:43.280404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9bff8c48-7a1b-4d96-bf5c-229d82ebaef5 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:43.289266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9bff8c48-7a1b-4d96-bf5c-229d82ebaef5 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:43.289808 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9bff8c48-7a1b-4d96-bf5c-229d82ebaef5 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:43.290646 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9bff8c48-7a1b-4d96-bf5c-229d82ebaef5 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:43.291092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9bff8c48-7a1b-4d96-bf5c-229d82ebaef5 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:43.385430 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f1ec0ef3-9b0a-4a26-99a2-1f671767c0d6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:43.386171 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f1ec0ef3-9b0a-4a26-99a2-1f671767c0d6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:43.394791 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f1ec0ef3-9b0a-4a26-99a2-1f671767c0d6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:43.395157 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f1ec0ef3-9b0a-4a26-99a2-1f671767c0d6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:43.395999 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f1ec0ef3-9b0a-4a26-99a2-1f671767c0d6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:43.396438 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f1ec0ef3-9b0a-4a26-99a2-1f671767c0d6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:43.431771 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9bff8c48-7a1b-4d96-bf5c-229d82ebaef5 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:43.441533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9bff8c48-7a1b-4d96-bf5c-229d82ebaef5 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.165604 Jun 10 13:08:43.442213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 291/559] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:43 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:43.478263 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f1ec0ef3-9b0a-4a26-99a2-1f671767c0d6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:43.485843 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f1ec0ef3-9b0a-4a26-99a2-1f671767c0d6 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1302 microversion: 2.1 time: 0.104625 Jun 10 13:08:43.486409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 269/560] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:43 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1302 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:43.968690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-96f92a7b-db45-4f98-b95d-8376f11c74a3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:43.969685 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-96f92a7b-db45-4f98-b95d-8376f11c74a3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:43.979730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96f92a7b-db45-4f98-b95d-8376f11c74a3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:43.985142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96f92a7b-db45-4f98-b95d-8376f11c74a3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:43.985142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96f92a7b-db45-4f98-b95d-8376f11c74a3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:43.985142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96f92a7b-db45-4f98-b95d-8376f11c74a3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:44.114272 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-540897825", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "networks": [{"uuid": "2ece6558-11f6-4246-abf0-a953a99b1ee4"}]}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:44.190813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-96f92a7b-db45-4f98-b95d-8376f11c74a3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:44.201679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-96f92a7b-db45-4f98-b95d-8376f11c74a3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.238057 Jun 10 13:08:44.202581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 292/561] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:43 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:44.226708 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.network.neutron [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] validate_networks() for [('2ece6558-11f6-4246-abf0-a953a99b1ee4', None, None, None, None, None)] {{(pid=118093) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:08:44.459982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-14d81a74-4081-46a5-a4f3-4adc6b11457f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:44.461204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-14d81a74-4081-46a5-a4f3-4adc6b11457f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:44.472526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-14d81a74-4081-46a5-a4f3-4adc6b11457f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:44.477058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-14d81a74-4081-46a5-a4f3-4adc6b11457f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:44.480003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-14d81a74-4081-46a5-a4f3-4adc6b11457f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:44.487106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-14d81a74-4081-46a5-a4f3-4adc6b11457f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:44.648372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-14d81a74-4081-46a5-a4f3-4adc6b11457f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:44.657666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-14d81a74-4081-46a5-a4f3-4adc6b11457f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.201298 Jun 10 13:08:44.659512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 293/562] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:44 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:44.664717 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e9c1631c-9694-4a6f-94fc-0eeda53e805c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:44.665542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e9c1631c-9694-4a6f-94fc-0eeda53e805c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:44.672894 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9c1631c-9694-4a6f-94fc-0eeda53e805c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:44.673616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9c1631c-9694-4a6f-94fc-0eeda53e805c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:44.674645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9c1631c-9694-4a6f-94fc-0eeda53e805c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:44.675235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9c1631c-9694-4a6f-94fc-0eeda53e805c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:44.842694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e9c1631c-9694-4a6f-94fc-0eeda53e805c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:44.852172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e9c1631c-9694-4a6f-94fc-0eeda53e805c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1370 microversion: 2.1 time: 0.191205 Jun 10 13:08:44.853045 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 294/563] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:44 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1370 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:44.964388 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Flavor limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:08:44.965085 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Image limits 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:08:44.965329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Flavor pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:08:44.965746 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Image pref 0:0:0 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:08:44.966175 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.virt.hardware [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118093) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:08:44.981327 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:44.991629 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:44.992986 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:44.993919 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:45.029887 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Getting quotas for project ff2616337ae440a89047fd9a50218387. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:08:45.035945 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Getting quotas for user 849e36f8653d4d1f9a7fb1df8fb913a9 and project ff2616337ae440a89047fd9a50218387. Resources: {'instances', 'ram', 'cores'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:08:45.048935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Going to run 1 instances... {{(pid=118093) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:08:45.051322 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118093) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:08:45.202130 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-890caed5-7259-42ae-bbe1-223ffe19bcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.092954 Jun 10 13:08:45.202819 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 270/564] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:08:44 2022] POST /compute/v2.1/servers => generated 380 bytes in 1095 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jun 10 13:08:45.222614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ea32db03-a7f9-4d42-b2da-33a7e576031b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:45.224644 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ea32db03-a7f9-4d42-b2da-33a7e576031b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:45.225915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5cdf009b-97cd-48f4-bfcc-efb1b9c5ef1a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:45.226793 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5cdf009b-97cd-48f4-bfcc-efb1b9c5ef1a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:45.237241 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ea32db03-a7f9-4d42-b2da-33a7e576031b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:45.237735 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ea32db03-a7f9-4d42-b2da-33a7e576031b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:45.238518 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ea32db03-a7f9-4d42-b2da-33a7e576031b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:45.239086 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ea32db03-a7f9-4d42-b2da-33a7e576031b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:45.323358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5cdf009b-97cd-48f4-bfcc-efb1b9c5ef1a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:45.331696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5cdf009b-97cd-48f4-bfcc-efb1b9c5ef1a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1218 microversion: 2.1 time: 0.117912 Jun 10 13:08:45.332306 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 295/565] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:45 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1218 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:45.437270 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-f16de6ba-52b9-4017-9bc1-d84fef7ba402 req-7a92bf02-7a9f-4da9-92da-85e7759dd290 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3baaefc5-1a36-4fa6-88d9-1764e5631a32", "name": "network-vif-plugged", "status": "completed", "tag": "72eeacbd-3288-4d28-90e4-93e208b2d17d"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:45.454463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-f16de6ba-52b9-4017-9bc1-d84fef7ba402 req-7a92bf02-7a9f-4da9-92da-85e7759dd290 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:45.457461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-f16de6ba-52b9-4017-9bc1-d84fef7ba402 req-7a92bf02-7a9f-4da9-92da-85e7759dd290 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:45.487018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-f16de6ba-52b9-4017-9bc1-d84fef7ba402 req-7a92bf02-7a9f-4da9-92da-85e7759dd290 service nova] Creating event network-vif-plugged:72eeacbd-3288-4d28-90e4-93e208b2d17d for instance 3baaefc5-1a36-4fa6-88d9-1764e5631a32 on hv-cinder-maste Jun 10 13:08:45.503212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-f16de6ba-52b9-4017-9bc1-d84fef7ba402 req-7a92bf02-7a9f-4da9-92da-85e7759dd290 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069775 Jun 10 13:08:45.503937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 296/566] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:08:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:08:45.513789 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ea32db03-a7f9-4d42-b2da-33a7e576031b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:45.525761 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ea32db03-a7f9-4d42-b2da-33a7e576031b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.309128 Jun 10 13:08:45.527023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 271/567] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:45 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:45.684289 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9a449181-861b-4cf1-9b61-f5391e297fff tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:45.685406 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9a449181-861b-4cf1-9b61-f5391e297fff tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:45.698820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a449181-861b-4cf1-9b61-f5391e297fff tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:45.699609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a449181-861b-4cf1-9b61-f5391e297fff tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:45.701305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a449181-861b-4cf1-9b61-f5391e297fff tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:45.701953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a449181-861b-4cf1-9b61-f5391e297fff tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:45.875041 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-701edd54-1039-4901-9f82-0eddd987eac8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:45.892868 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-701edd54-1039-4901-9f82-0eddd987eac8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:45.910457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9a449181-861b-4cf1-9b61-f5391e297fff tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:45.917659 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-701edd54-1039-4901-9f82-0eddd987eac8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:45.917966 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-701edd54-1039-4901-9f82-0eddd987eac8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:45.922605 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9a449181-861b-4cf1-9b61-f5391e297fff tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.242714 Jun 10 13:08:45.923025 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-701edd54-1039-4901-9f82-0eddd987eac8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:45.923555 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-701edd54-1039-4901-9f82-0eddd987eac8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:45.924415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 297/568] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:45 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:45.948152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-9d9e82a0-5c54-4369-887f-c817f45faaa0 req-c46bc2b9-2af0-4ae8-af7c-8ae01ae06637 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c699e532-a2dd-4fb8-9ba9-0e7089eff32f", "tag": "9d09560d-d177-4fe5-bce5-a1dc351d5f14"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:45.959452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-9d9e82a0-5c54-4369-887f-c817f45faaa0 req-c46bc2b9-2af0-4ae8-af7c-8ae01ae06637 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:45.960851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-9d9e82a0-5c54-4369-887f-c817f45faaa0 req-c46bc2b9-2af0-4ae8-af7c-8ae01ae06637 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:45.987646 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-9d9e82a0-5c54-4369-887f-c817f45faaa0 req-c46bc2b9-2af0-4ae8-af7c-8ae01ae06637 service nova] Creating event network-changed:9d09560d-d177-4fe5-bce5-a1dc351d5f14 for instance c699e532-a2dd-4fb8-9ba9-0e7089eff32f on hv-cinder-maste Jun 10 13:08:45.993077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-9d9e82a0-5c54-4369-887f-c817f45faaa0 req-c46bc2b9-2af0-4ae8-af7c-8ae01ae06637 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055966 Jun 10 13:08:45.994116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 298/569] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:08:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:08:46.343184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-701edd54-1039-4901-9f82-0eddd987eac8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:46.355689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bcdf9ae1-c1d6-4ab9-ad7d-10d58710bae7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:46.359837 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bcdf9ae1-c1d6-4ab9-ad7d-10d58710bae7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:46.369915 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-701edd54-1039-4901-9f82-0eddd987eac8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1370 microversion: 2.1 time: 0.502873 Jun 10 13:08:46.371837 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 272/570] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:45 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1370 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:46.383976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bcdf9ae1-c1d6-4ab9-ad7d-10d58710bae7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:46.384649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bcdf9ae1-c1d6-4ab9-ad7d-10d58710bae7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:46.385619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bcdf9ae1-c1d6-4ab9-ad7d-10d58710bae7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:46.386213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bcdf9ae1-c1d6-4ab9-ad7d-10d58710bae7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:46.456210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bcdf9ae1-c1d6-4ab9-ad7d-10d58710bae7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:46.464308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bcdf9ae1-c1d6-4ab9-ad7d-10d58710bae7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1222 microversion: 2.1 time: 0.119303 Jun 10 13:08:46.474152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 299/571] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:46 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1222 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:46.544116 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ef1b85fc-3b2b-46e1-b468-93c714f628e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:46.544718 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ef1b85fc-3b2b-46e1-b468-93c714f628e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:46.547176 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:46.552026 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef1b85fc-3b2b-46e1-b468-93c714f628e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:46.552472 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef1b85fc-3b2b-46e1-b468-93c714f628e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:46.553412 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef1b85fc-3b2b-46e1-b468-93c714f628e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:46.553842 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef1b85fc-3b2b-46e1-b468-93c714f628e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:46.760287 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ef1b85fc-3b2b-46e1-b468-93c714f628e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:46.773787 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ef1b85fc-3b2b-46e1-b468-93c714f628e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.233527 Jun 10 13:08:46.774766 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 273/572] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:46 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:46.941689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-314763f8-99e0-4f7a-8764-17e36d75171b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:46.942320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-314763f8-99e0-4f7a-8764-17e36d75171b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:46.949298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-314763f8-99e0-4f7a-8764-17e36d75171b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:46.950210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-314763f8-99e0-4f7a-8764-17e36d75171b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:46.952009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-314763f8-99e0-4f7a-8764-17e36d75171b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:46.952816 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-314763f8-99e0-4f7a-8764-17e36d75171b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:47.119563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-314763f8-99e0-4f7a-8764-17e36d75171b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:47.127925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-314763f8-99e0-4f7a-8764-17e36d75171b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1678 microversion: 2.1 time: 0.190933 Jun 10 13:08:47.138043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 300/573] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:46 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1678 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:47.387586 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0d13890c-6173-4250-ba2e-676682e21a58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:47.388328 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0d13890c-6173-4250-ba2e-676682e21a58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:47.395943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0d13890c-6173-4250-ba2e-676682e21a58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:47.396625 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0d13890c-6173-4250-ba2e-676682e21a58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:47.397429 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0d13890c-6173-4250-ba2e-676682e21a58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:47.397864 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0d13890c-6173-4250-ba2e-676682e21a58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:47.481568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1d00eb5d-eb1d-47c1-9e8d-331969fdcc7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:47.482336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1d00eb5d-eb1d-47c1-9e8d-331969fdcc7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:47.489057 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d00eb5d-eb1d-47c1-9e8d-331969fdcc7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:47.489669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d00eb5d-eb1d-47c1-9e8d-331969fdcc7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:47.490492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d00eb5d-eb1d-47c1-9e8d-331969fdcc7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:47.490988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d00eb5d-eb1d-47c1-9e8d-331969fdcc7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:47.559531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1d00eb5d-eb1d-47c1-9e8d-331969fdcc7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:47.561672 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0d13890c-6173-4250-ba2e-676682e21a58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:47.567466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1d00eb5d-eb1d-47c1-9e8d-331969fdcc7c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1222 microversion: 2.1 time: 0.090029 Jun 10 13:08:47.568040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 301/574] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:47 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1222 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:47.575948 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0d13890c-6173-4250-ba2e-676682e21a58 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1370 microversion: 2.1 time: 0.192169 Jun 10 13:08:47.576533 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 274/575] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:47 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1370 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:47.791350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8d1c4a8a-7d0c-4e3c-be5f-14f655d0fdee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:47.792457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8d1c4a8a-7d0c-4e3c-be5f-14f655d0fdee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:47.799309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8d1c4a8a-7d0c-4e3c-be5f-14f655d0fdee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:47.800146 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8d1c4a8a-7d0c-4e3c-be5f-14f655d0fdee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:47.801348 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8d1c4a8a-7d0c-4e3c-be5f-14f655d0fdee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:47.802055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8d1c4a8a-7d0c-4e3c-be5f-14f655d0fdee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:47.832048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:47.971194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8d1c4a8a-7d0c-4e3c-be5f-14f655d0fdee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:47.980242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8d1c4a8a-7d0c-4e3c-be5f-14f655d0fdee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1260 microversion: 2.1 time: 0.193104 Jun 10 13:08:47.988827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 302/576] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:47 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1260 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:48.148667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7068bd02-511e-4ae5-b052-b08529032e9a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:48.149325 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7068bd02-511e-4ae5-b052-b08529032e9a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:48.156657 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7068bd02-511e-4ae5-b052-b08529032e9a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:48.157544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7068bd02-511e-4ae5-b052-b08529032e9a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:48.158615 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7068bd02-511e-4ae5-b052-b08529032e9a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:48.159276 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7068bd02-511e-4ae5-b052-b08529032e9a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:48.308532 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7068bd02-511e-4ae5-b052-b08529032e9a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:48.317273 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7068bd02-511e-4ae5-b052-b08529032e9a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1655 microversion: 2.1 time: 0.175705 Jun 10 13:08:48.317851 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 275/577] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:48 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1655 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:48.344655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e61fbeee-b57f-462c-89fb-7ba8dea8ba91 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:48.351767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e61fbeee-b57f-462c-89fb-7ba8dea8ba91 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:48.352423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e61fbeee-b57f-462c-89fb-7ba8dea8ba91 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:48.398130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e61fbeee-b57f-462c-89fb-7ba8dea8ba91 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Going to try to start instance {{(pid=118092) start /opt/stack/nova/nova/compute/api.py:2730}} Jun 10 13:08:48.586272 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e61fbeee-b57f-462c-89fb-7ba8dea8ba91 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "POST /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/action" status: 202 len: 0 microversion: 2.1 time: 0.245440 Jun 10 13:08:48.591876 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e036ac6b-5a4c-44a2-9f75-c5e304d4fcd2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:48.592345 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e036ac6b-5a4c-44a2-9f75-c5e304d4fcd2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:48.592987 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 303/578] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Jun 10 13:08:48 2022] POST /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/action => generated 0 bytes in 263 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jun 10 13:08:48.599393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6c1b6dde-1a13-405a-8aed-a2612d0a6879 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:48.600318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6c1b6dde-1a13-405a-8aed-a2612d0a6879 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:48.601471 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e036ac6b-5a4c-44a2-9f75-c5e304d4fcd2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:48.602122 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e036ac6b-5a4c-44a2-9f75-c5e304d4fcd2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:48.603099 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e036ac6b-5a4c-44a2-9f75-c5e304d4fcd2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:48.603523 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e036ac6b-5a4c-44a2-9f75-c5e304d4fcd2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:48.607213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c1b6dde-1a13-405a-8aed-a2612d0a6879 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:48.607661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c1b6dde-1a13-405a-8aed-a2612d0a6879 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:48.608444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c1b6dde-1a13-405a-8aed-a2612d0a6879 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:48.608901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c1b6dde-1a13-405a-8aed-a2612d0a6879 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:48.701137 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e036ac6b-5a4c-44a2-9f75-c5e304d4fcd2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:48.711616 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e036ac6b-5a4c-44a2-9f75-c5e304d4fcd2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1214 microversion: 2.1 time: 0.130821 Jun 10 13:08:48.712206 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 276/579] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:48 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1214 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:48.718004 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-115fc505-93a4-4dc9-99dd-42962d784446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:48.718847 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-115fc505-93a4-4dc9-99dd-42962d784446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:48.727949 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-115fc505-93a4-4dc9-99dd-42962d784446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:48.728511 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-115fc505-93a4-4dc9-99dd-42962d784446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:48.729492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-115fc505-93a4-4dc9-99dd-42962d784446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:48.730083 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-115fc505-93a4-4dc9-99dd-42962d784446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:48.733831 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:48.790969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6c1b6dde-1a13-405a-8aed-a2612d0a6879 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:48.802149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6c1b6dde-1a13-405a-8aed-a2612d0a6879 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1370 microversion: 2.1 time: 0.207717 Jun 10 13:08:48.804370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 304/580] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:48 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1370 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:48.914810 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-115fc505-93a4-4dc9-99dd-42962d784446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:48.923690 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-115fc505-93a4-4dc9-99dd-42962d784446 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1664 microversion: 2.1 time: 0.210157 Jun 10 13:08:48.924558 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 277/581] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:48 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1664 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:48.997725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-55b75c82-c6ca-484b-8374-8604c461784d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:48.998258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-55b75c82-c6ca-484b-8374-8604c461784d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:49.010439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55b75c82-c6ca-484b-8374-8604c461784d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:49.011242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55b75c82-c6ca-484b-8374-8604c461784d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:49.014361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55b75c82-c6ca-484b-8374-8604c461784d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:49.014361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55b75c82-c6ca-484b-8374-8604c461784d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:49.194113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-55b75c82-c6ca-484b-8374-8604c461784d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:49.204035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-55b75c82-c6ca-484b-8374-8604c461784d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1445 microversion: 2.1 time: 0.210342 Jun 10 13:08:49.216793 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 305/582] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:48 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1445 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:49.226667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c1fb5746-171d-4a2f-978d-3a97cbfb11f8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:49.227296 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c1fb5746-171d-4a2f-978d-3a97cbfb11f8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:49.243336 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c1fb5746-171d-4a2f-978d-3a97cbfb11f8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:49.244927 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c1fb5746-171d-4a2f-978d-3a97cbfb11f8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:49.247977 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c1fb5746-171d-4a2f-978d-3a97cbfb11f8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:49.248899 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c1fb5746-171d-4a2f-978d-3a97cbfb11f8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:49.724195 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c1fb5746-171d-4a2f-978d-3a97cbfb11f8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:49.737327 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5e8b584a-6a35-432f-bdea-8bc6f0c237d5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:49.738152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5e8b584a-6a35-432f-bdea-8bc6f0c237d5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:49.739260 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c1fb5746-171d-4a2f-978d-3a97cbfb11f8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1445 microversion: 2.1 time: 0.516603 Jun 10 13:08:49.739914 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 278/583] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:49 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1445 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:49.744454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e8b584a-6a35-432f-bdea-8bc6f0c237d5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:49.745191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e8b584a-6a35-432f-bdea-8bc6f0c237d5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:49.746320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e8b584a-6a35-432f-bdea-8bc6f0c237d5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:49.746832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e8b584a-6a35-432f-bdea-8bc6f0c237d5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:49.821733 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-858d3dab-bf43-4d0f-a047-edf65900d0f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:49.822298 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-858d3dab-bf43-4d0f-a047-edf65900d0f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:49.825740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5e8b584a-6a35-432f-bdea-8bc6f0c237d5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:49.835613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5e8b584a-6a35-432f-bdea-8bc6f0c237d5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1270 microversion: 2.1 time: 0.103531 Jun 10 13:08:49.836713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 306/584] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:49 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1270 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:49.837129 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-858d3dab-bf43-4d0f-a047-edf65900d0f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:49.837963 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-858d3dab-bf43-4d0f-a047-edf65900d0f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:49.838343 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-858d3dab-bf43-4d0f-a047-edf65900d0f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:49.838804 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-858d3dab-bf43-4d0f-a047-edf65900d0f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:49.945971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4a5e39de-eabf-4c26-baf4-ce69ed4bee9f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:49.947621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4a5e39de-eabf-4c26-baf4-ce69ed4bee9f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:49.956970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a5e39de-eabf-4c26-baf4-ce69ed4bee9f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:49.957439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a5e39de-eabf-4c26-baf4-ce69ed4bee9f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:49.958219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a5e39de-eabf-4c26-baf4-ce69ed4bee9f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:49.958665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a5e39de-eabf-4c26-baf4-ce69ed4bee9f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:50.182536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4a5e39de-eabf-4c26-baf4-ce69ed4bee9f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:50.191259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4a5e39de-eabf-4c26-baf4-ce69ed4bee9f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1664 microversion: 2.1 time: 0.253716 Jun 10 13:08:50.201821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 307/585] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:49 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1664 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:50.221245 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-858d3dab-bf43-4d0f-a047-edf65900d0f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:50.237775 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-858d3dab-bf43-4d0f-a047-edf65900d0f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1370 microversion: 2.1 time: 0.420033 Jun 10 13:08:50.240827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 279/586] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:49 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1370 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:50.853857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-df5d34e4-b8ee-4fb1-af4d-4b3e8415b79f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:50.865445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-df5d34e4-b8ee-4fb1-af4d-4b3e8415b79f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:50.865445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df5d34e4-b8ee-4fb1-af4d-4b3e8415b79f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:50.865445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df5d34e4-b8ee-4fb1-af4d-4b3e8415b79f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:50.865445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df5d34e4-b8ee-4fb1-af4d-4b3e8415b79f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:50.865884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df5d34e4-b8ee-4fb1-af4d-4b3e8415b79f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:50.940740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-df5d34e4-b8ee-4fb1-af4d-4b3e8415b79f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:50.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-df5d34e4-b8ee-4fb1-af4d-4b3e8415b79f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1288 microversion: 2.1 time: 0.098608 Jun 10 13:08:50.956535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 308/587] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:50 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1288 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:51.209565 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3f8f5f39-402a-47cf-8f4b-05336e7e49a1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:51.210407 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3f8f5f39-402a-47cf-8f4b-05336e7e49a1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:51.221831 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3f8f5f39-402a-47cf-8f4b-05336e7e49a1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:51.224263 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3f8f5f39-402a-47cf-8f4b-05336e7e49a1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:51.225308 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3f8f5f39-402a-47cf-8f4b-05336e7e49a1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:51.231451 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3f8f5f39-402a-47cf-8f4b-05336e7e49a1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:51.255958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b20a4772-dc51-4ec9-a359-ba03e68f5817 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:51.256562 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b20a4772-dc51-4ec9-a359-ba03e68f5817 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:51.264952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b20a4772-dc51-4ec9-a359-ba03e68f5817 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:51.265656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b20a4772-dc51-4ec9-a359-ba03e68f5817 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:51.268520 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b20a4772-dc51-4ec9-a359-ba03e68f5817 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:51.269353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b20a4772-dc51-4ec9-a359-ba03e68f5817 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:51.623791 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3f8f5f39-402a-47cf-8f4b-05336e7e49a1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:51.645235 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3f8f5f39-402a-47cf-8f4b-05336e7e49a1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1664 microversion: 2.1 time: 0.439157 Jun 10 13:08:51.661116 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 280/588] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:51 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1664 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:51.674645 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-b3276b80-d4f2-4409-bc92-babd621dd6f9 req-28c10efe-639f-4249-8997-b4364a2a6aa7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c699e532-a2dd-4fb8-9ba9-0e7089eff32f", "name": "network-vif-plugged", "status": "completed", "tag": "9d09560d-d177-4fe5-bce5-a1dc351d5f14"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:51.710197 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-b3276b80-d4f2-4409-bc92-babd621dd6f9 req-28c10efe-639f-4249-8997-b4364a2a6aa7 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:51.711722 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-b3276b80-d4f2-4409-bc92-babd621dd6f9 req-28c10efe-639f-4249-8997-b4364a2a6aa7 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:51.729043 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-b3276b80-d4f2-4409-bc92-babd621dd6f9 req-28c10efe-639f-4249-8997-b4364a2a6aa7 service nova] Creating event network-vif-plugged:9d09560d-d177-4fe5-bce5-a1dc351d5f14 for instance c699e532-a2dd-4fb8-9ba9-0e7089eff32f on hv-cinder-maste Jun 10 13:08:51.736576 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-b3276b80-d4f2-4409-bc92-babd621dd6f9 req-28c10efe-639f-4249-8997-b4364a2a6aa7 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065696 Jun 10 13:08:51.737713 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 281/589] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:08:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:08:51.900454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b20a4772-dc51-4ec9-a359-ba03e68f5817 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:51.917490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b20a4772-dc51-4ec9-a359-ba03e68f5817 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1370 microversion: 2.1 time: 0.665058 Jun 10 13:08:51.928280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 309/590] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:51 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1370 bytes in 677 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:51.970081 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a6100106-35c2-4474-b8cd-7251a4ffa95f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:51.971472 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a6100106-35c2-4474-b8cd-7251a4ffa95f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:51.991533 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a6100106-35c2-4474-b8cd-7251a4ffa95f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:51.993924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a6100106-35c2-4474-b8cd-7251a4ffa95f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:52.000502 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a6100106-35c2-4474-b8cd-7251a4ffa95f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:52.001324 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a6100106-35c2-4474-b8cd-7251a4ffa95f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:52.130330 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a6100106-35c2-4474-b8cd-7251a4ffa95f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:52.149831 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a6100106-35c2-4474-b8cd-7251a4ffa95f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1276 microversion: 2.1 time: 0.181171 Jun 10 13:08:52.153112 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 282/591] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:51 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1276 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:52.674902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c26a4c25-71af-43c7-858e-e3abcc7fb27d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:52.675695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c26a4c25-71af-43c7-858e-e3abcc7fb27d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:52.682798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c26a4c25-71af-43c7-858e-e3abcc7fb27d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:52.683369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c26a4c25-71af-43c7-858e-e3abcc7fb27d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:52.684529 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c26a4c25-71af-43c7-858e-e3abcc7fb27d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:52.685108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c26a4c25-71af-43c7-858e-e3abcc7fb27d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:52.937753 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-310f9ebb-2515-44fd-bb42-8ed41ae06c72 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:52.939060 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-310f9ebb-2515-44fd-bb42-8ed41ae06c72 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:52.946328 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-310f9ebb-2515-44fd-bb42-8ed41ae06c72 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:52.947140 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-310f9ebb-2515-44fd-bb42-8ed41ae06c72 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:52.948133 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-310f9ebb-2515-44fd-bb42-8ed41ae06c72 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:52.948566 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-310f9ebb-2515-44fd-bb42-8ed41ae06c72 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:52.964149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c26a4c25-71af-43c7-858e-e3abcc7fb27d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:52.979407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c26a4c25-71af-43c7-858e-e3abcc7fb27d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1664 microversion: 2.1 time: 0.308293 Jun 10 13:08:52.980800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 310/592] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:52 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1664 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:53.172560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d264b101-fa75-4fd5-a103-44b64cb134ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:53.173304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d264b101-fa75-4fd5-a103-44b64cb134ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:53.185371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d264b101-fa75-4fd5-a103-44b64cb134ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:53.186175 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d264b101-fa75-4fd5-a103-44b64cb134ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:53.187429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d264b101-fa75-4fd5-a103-44b64cb134ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:53.188086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d264b101-fa75-4fd5-a103-44b64cb134ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:53.372755 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-310f9ebb-2515-44fd-bb42-8ed41ae06c72 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:53.391517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-310f9ebb-2515-44fd-bb42-8ed41ae06c72 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1370 microversion: 2.1 time: 0.460592 Jun 10 13:08:53.392704 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 283/593] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:52 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1370 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:08:53.432260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d264b101-fa75-4fd5-a103-44b64cb134ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:53.444489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d264b101-fa75-4fd5-a103-44b64cb134ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1318 microversion: 2.1 time: 0.276869 Jun 10 13:08:53.447753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 311/594] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:53 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1318 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:53.832756 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-5044d794-d496-41bf-84b5-a3b85b96b5e5 req-ec8d8f30-cd21-4076-b538-171d91cd0aa1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3baaefc5-1a36-4fa6-88d9-1764e5631a32", "tag": "72eeacbd-3288-4d28-90e4-93e208b2d17d"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:53.850084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-5044d794-d496-41bf-84b5-a3b85b96b5e5 req-ec8d8f30-cd21-4076-b538-171d91cd0aa1 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:53.850795 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-5044d794-d496-41bf-84b5-a3b85b96b5e5 req-ec8d8f30-cd21-4076-b538-171d91cd0aa1 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:53.879333 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-5044d794-d496-41bf-84b5-a3b85b96b5e5 req-ec8d8f30-cd21-4076-b538-171d91cd0aa1 service nova] Creating event network-changed:72eeacbd-3288-4d28-90e4-93e208b2d17d for instance 3baaefc5-1a36-4fa6-88d9-1764e5631a32 on hv-cinder-maste Jun 10 13:08:53.898595 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-5044d794-d496-41bf-84b5-a3b85b96b5e5 req-ec8d8f30-cd21-4076-b538-171d91cd0aa1 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.159514 Jun 10 13:08:53.899585 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 284/595] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:08:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:08:53.999037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-04aba63d-f0a2-4cbf-b78b-c7f127c54e2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:54.001342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-04aba63d-f0a2-4cbf-b78b-c7f127c54e2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:54.010010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04aba63d-f0a2-4cbf-b78b-c7f127c54e2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:54.011761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04aba63d-f0a2-4cbf-b78b-c7f127c54e2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:54.013480 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04aba63d-f0a2-4cbf-b78b-c7f127c54e2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:54.014097 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04aba63d-f0a2-4cbf-b78b-c7f127c54e2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:54.196720 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-04aba63d-f0a2-4cbf-b78b-c7f127c54e2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:54.205058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-04aba63d-f0a2-4cbf-b78b-c7f127c54e2c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1668 microversion: 2.1 time: 0.210760 Jun 10 13:08:54.216201 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 312/596] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:53 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1668 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:54.310854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-2bdb3a53-2b6b-4f65-8358-9f98381f6d68 req-731a0c2d-a619-4c3b-af2e-69a1b373e25c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744", "tag": "46531dd1-3591-49bf-b170-0a5b54b53fcd"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:54.323314 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-2bdb3a53-2b6b-4f65-8358-9f98381f6d68 req-731a0c2d-a619-4c3b-af2e-69a1b373e25c service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:54.326008 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-2bdb3a53-2b6b-4f65-8358-9f98381f6d68 req-731a0c2d-a619-4c3b-af2e-69a1b373e25c service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:54.352161 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-2bdb3a53-2b6b-4f65-8358-9f98381f6d68 req-731a0c2d-a619-4c3b-af2e-69a1b373e25c service nova] Creating event network-changed:46531dd1-3591-49bf-b170-0a5b54b53fcd for instance a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 on hv-cinder-maste Jun 10 13:08:54.360242 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-2bdb3a53-2b6b-4f65-8358-9f98381f6d68 req-731a0c2d-a619-4c3b-af2e-69a1b373e25c service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053735 Jun 10 13:08:54.364010 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 285/597] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:08:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:08:54.410088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-48f277ea-eb6d-4501-af07-b06e21e60629 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:54.410732 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-48f277ea-eb6d-4501-af07-b06e21e60629 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:54.420110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48f277ea-eb6d-4501-af07-b06e21e60629 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:54.425982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48f277ea-eb6d-4501-af07-b06e21e60629 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:54.427067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48f277ea-eb6d-4501-af07-b06e21e60629 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:54.427662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48f277ea-eb6d-4501-af07-b06e21e60629 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:54.491248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3b0da51d-cdac-4d73-b062-b2b13569e222 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:54.492638 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3b0da51d-cdac-4d73-b062-b2b13569e222 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:54.505790 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3b0da51d-cdac-4d73-b062-b2b13569e222 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:54.506700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3b0da51d-cdac-4d73-b062-b2b13569e222 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:54.507679 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3b0da51d-cdac-4d73-b062-b2b13569e222 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:54.508202 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3b0da51d-cdac-4d73-b062-b2b13569e222 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:54.624931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-48f277ea-eb6d-4501-af07-b06e21e60629 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:54.635404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-48f277ea-eb6d-4501-af07-b06e21e60629 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1370 microversion: 2.1 time: 0.229957 Jun 10 13:08:54.648052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 313/598] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:54 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1370 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:54.750716 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3b0da51d-cdac-4d73-b062-b2b13569e222 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:54.760025 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3b0da51d-cdac-4d73-b062-b2b13569e222 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1318 microversion: 2.1 time: 0.273578 Jun 10 13:08:54.760693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 286/599] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:54 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1318 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:55.653446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-679bd8ec-d5b7-4496-872f-8ad981c0192c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:55.654678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-679bd8ec-d5b7-4496-872f-8ad981c0192c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:55.663416 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=4 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:55.665492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-679bd8ec-d5b7-4496-872f-8ad981c0192c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:55.668329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-679bd8ec-d5b7-4496-872f-8ad981c0192c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:55.668329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-679bd8ec-d5b7-4496-872f-8ad981c0192c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:55.668329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-679bd8ec-d5b7-4496-872f-8ad981c0192c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:55.778004 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-959a8114-0755-4b6e-adaf-553833427726 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:55.778611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-959a8114-0755-4b6e-adaf-553833427726 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:55.780581 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=4 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:55.781287 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=16 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:55.789451 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-959a8114-0755-4b6e-adaf-553833427726 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:55.810815 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-959a8114-0755-4b6e-adaf-553833427726 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:55.810815 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-959a8114-0755-4b6e-adaf-553833427726 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:55.810815 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-959a8114-0755-4b6e-adaf-553833427726 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:55.855299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-679bd8ec-d5b7-4496-872f-8ad981c0192c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:55.864011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-679bd8ec-d5b7-4496-872f-8ad981c0192c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1564 microversion: 2.1 time: 0.214260 Jun 10 13:08:55.864756 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 314/600] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:55 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1564 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:55.881612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-37a19e0e-f5af-4c3b-8fc7-ac8cb2dcb39b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:55.882104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-37a19e0e-f5af-4c3b-8fc7-ac8cb2dcb39b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:55.888421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-37a19e0e-f5af-4c3b-8fc7-ac8cb2dcb39b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:55.888848 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-37a19e0e-f5af-4c3b-8fc7-ac8cb2dcb39b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:55.889654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-37a19e0e-f5af-4c3b-8fc7-ac8cb2dcb39b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:55.890091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-37a19e0e-f5af-4c3b-8fc7-ac8cb2dcb39b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:55.974850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-37a19e0e-f5af-4c3b-8fc7-ac8cb2dcb39b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.097142 Jun 10 13:08:55.975463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 315/601] 10.222.0.25 () {58 vars in 1274 bytes} [Fri Jun 10 13:08:55 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f/os-interface => generated 284 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:08:56.036493 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-959a8114-0755-4b6e-adaf-553833427726 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:56.044949 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-959a8114-0755-4b6e-adaf-553833427726 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1318 microversion: 2.1 time: 0.271008 Jun 10 13:08:56.045520 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 287/602] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:55 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1318 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:57.062859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5db922f1-282b-4036-aa34-f597f6b21a24 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:57.063470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5db922f1-282b-4036-aa34-f597f6b21a24 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:57.070041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5db922f1-282b-4036-aa34-f597f6b21a24 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:57.070485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5db922f1-282b-4036-aa34-f597f6b21a24 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:57.071245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5db922f1-282b-4036-aa34-f597f6b21a24 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:57.071854 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5db922f1-282b-4036-aa34-f597f6b21a24 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:57.196300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-d20eec3a-3897-4798-8291-8d1ea94b95be req-8918c91e-89cb-4b75-ad77-73b93060aa84 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c699e532-a2dd-4fb8-9ba9-0e7089eff32f", "tag": "9d09560d-d177-4fe5-bce5-a1dc351d5f14"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:08:57.206209 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-d20eec3a-3897-4798-8291-8d1ea94b95be req-8918c91e-89cb-4b75-ad77-73b93060aa84 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:57.206622 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-d20eec3a-3897-4798-8291-8d1ea94b95be req-8918c91e-89cb-4b75-ad77-73b93060aa84 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:57.218063 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-d20eec3a-3897-4798-8291-8d1ea94b95be req-8918c91e-89cb-4b75-ad77-73b93060aa84 service nova] Creating event network-changed:9d09560d-d177-4fe5-bce5-a1dc351d5f14 for instance c699e532-a2dd-4fb8-9ba9-0e7089eff32f on hv-cinder-maste Jun 10 13:08:57.230980 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-d20eec3a-3897-4798-8291-8d1ea94b95be req-8918c91e-89cb-4b75-ad77-73b93060aa84 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038389 Jun 10 13:08:57.234081 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 288/603] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:08:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:08:57.246252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5db922f1-282b-4036-aa34-f597f6b21a24 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:57.254771 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5db922f1-282b-4036-aa34-f597f6b21a24 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1318 microversion: 2.1 time: 0.196212 Jun 10 13:08:57.259873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 316/604] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:57 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1318 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:58.274016 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0799f441-97cf-41e6-8c42-476257dfb0d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:58.274868 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0799f441-97cf-41e6-8c42-476257dfb0d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:58.281359 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0799f441-97cf-41e6-8c42-476257dfb0d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:58.281997 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0799f441-97cf-41e6-8c42-476257dfb0d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:58.282781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0799f441-97cf-41e6-8c42-476257dfb0d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:58.283220 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0799f441-97cf-41e6-8c42-476257dfb0d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:58.415361 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0799f441-97cf-41e6-8c42-476257dfb0d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:58.423062 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0799f441-97cf-41e6-8c42-476257dfb0d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1318 microversion: 2.1 time: 0.155043 Jun 10 13:08:58.426804 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 289/605] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:58 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1318 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:08:59.440291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3ec5f15f-8c64-4d2c-9ef7-f1eee3bc2174 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:08:59.440920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3ec5f15f-8c64-4d2c-9ef7-f1eee3bc2174 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:08:59.447450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3ec5f15f-8c64-4d2c-9ef7-f1eee3bc2174 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:59.451183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3ec5f15f-8c64-4d2c-9ef7-f1eee3bc2174 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:59.451183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3ec5f15f-8c64-4d2c-9ef7-f1eee3bc2174 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:08:59.451183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3ec5f15f-8c64-4d2c-9ef7-f1eee3bc2174 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:08:59.576922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3ec5f15f-8c64-4d2c-9ef7-f1eee3bc2174 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:08:59.584349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3ec5f15f-8c64-4d2c-9ef7-f1eee3bc2174 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1318 microversion: 2.1 time: 0.148664 Jun 10 13:08:59.584953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 317/606] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:08:59 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1318 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:09:00.599935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a9791ea9-3aa7-4544-8bca-0a470ae0e5e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:00.600591 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a9791ea9-3aa7-4544-8bca-0a470ae0e5e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:00.607261 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a9791ea9-3aa7-4544-8bca-0a470ae0e5e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:00.607790 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a9791ea9-3aa7-4544-8bca-0a470ae0e5e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:00.608558 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a9791ea9-3aa7-4544-8bca-0a470ae0e5e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:00.609016 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a9791ea9-3aa7-4544-8bca-0a470ae0e5e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:00.737326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a9791ea9-3aa7-4544-8bca-0a470ae0e5e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:00.744597 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a9791ea9-3aa7-4544-8bca-0a470ae0e5e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1318 microversion: 2.1 time: 0.148475 Jun 10 13:09:00.749988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 290/607] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:00 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1318 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:09:01.553977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-8cbe364d-a736-41d8-a360-4e2a651992dc req-1c37975e-d7cc-487b-b543-0d93a4f93aed service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744", "name": "network-vif-plugged", "status": "completed", "tag": "46531dd1-3591-49bf-b170-0a5b54b53fcd"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:09:01.562089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-8cbe364d-a736-41d8-a360-4e2a651992dc req-1c37975e-d7cc-487b-b543-0d93a4f93aed service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:01.562522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-8cbe364d-a736-41d8-a360-4e2a651992dc req-1c37975e-d7cc-487b-b543-0d93a4f93aed service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:01.573321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-8cbe364d-a736-41d8-a360-4e2a651992dc req-1c37975e-d7cc-487b-b543-0d93a4f93aed service nova] Creating event network-vif-plugged:46531dd1-3591-49bf-b170-0a5b54b53fcd for instance a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 on hv-cinder-maste Jun 10 13:09:01.578216 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-8cbe364d-a736-41d8-a360-4e2a651992dc req-1c37975e-d7cc-487b-b543-0d93a4f93aed service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.027910 Jun 10 13:09:01.578804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 318/608] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:09:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:09:01.762264 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f0b86aaf-8987-4485-bba4-5b1f62ba8d52 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:01.762871 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f0b86aaf-8987-4485-bba4-5b1f62ba8d52 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:01.770023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f0b86aaf-8987-4485-bba4-5b1f62ba8d52 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:01.770467 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f0b86aaf-8987-4485-bba4-5b1f62ba8d52 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:01.771218 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f0b86aaf-8987-4485-bba4-5b1f62ba8d52 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:01.771643 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f0b86aaf-8987-4485-bba4-5b1f62ba8d52 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:01.907387 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f0b86aaf-8987-4485-bba4-5b1f62ba8d52 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:01.915761 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f0b86aaf-8987-4485-bba4-5b1f62ba8d52 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1318 microversion: 2.1 time: 0.158173 Jun 10 13:09:01.917374 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 291/609] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:01 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1318 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:09:02.935258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b7ab4438-8f52-45f0-a5d6-5ac1cb9275da tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:02.935985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b7ab4438-8f52-45f0-a5d6-5ac1cb9275da tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:02.942244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b7ab4438-8f52-45f0-a5d6-5ac1cb9275da tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:02.942691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b7ab4438-8f52-45f0-a5d6-5ac1cb9275da tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:02.943449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b7ab4438-8f52-45f0-a5d6-5ac1cb9275da tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:02.943882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b7ab4438-8f52-45f0-a5d6-5ac1cb9275da tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:02.946521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:03.074889 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b7ab4438-8f52-45f0-a5d6-5ac1cb9275da tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:03.082388 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b7ab4438-8f52-45f0-a5d6-5ac1cb9275da tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1318 microversion: 2.1 time: 0.151942 Jun 10 13:09:03.082985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 319/610] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:02 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1318 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:09:04.100042 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-75e81b00-792e-4cc1-a7fd-a41010f63fab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:04.100653 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-75e81b00-792e-4cc1-a7fd-a41010f63fab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:04.108001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-75e81b00-792e-4cc1-a7fd-a41010f63fab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:04.108492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-75e81b00-792e-4cc1-a7fd-a41010f63fab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:04.109283 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-75e81b00-792e-4cc1-a7fd-a41010f63fab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:04.109935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-75e81b00-792e-4cc1-a7fd-a41010f63fab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:04.241228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-75e81b00-792e-4cc1-a7fd-a41010f63fab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:04.249345 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-75e81b00-792e-4cc1-a7fd-a41010f63fab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1318 microversion: 2.1 time: 0.154096 Jun 10 13:09:04.249918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 292/611] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:04 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1318 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:09:05.267074 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2c5f1ea5-8705-4649-a79e-3f8263dc8263 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:05.267700 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2c5f1ea5-8705-4649-a79e-3f8263dc8263 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:05.275486 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2c5f1ea5-8705-4649-a79e-3f8263dc8263 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:05.275927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2c5f1ea5-8705-4649-a79e-3f8263dc8263 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:05.276681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2c5f1ea5-8705-4649-a79e-3f8263dc8263 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:05.277137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2c5f1ea5-8705-4649-a79e-3f8263dc8263 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:05.412745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2c5f1ea5-8705-4649-a79e-3f8263dc8263 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:05.419666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2c5f1ea5-8705-4649-a79e-3f8263dc8263 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1507 microversion: 2.1 time: 0.156791 Jun 10 13:09:05.425317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 320/612] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:05 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1507 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:09:05.439492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-bff2528a-5991-4f1d-8e92-b6e15e8701ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "802588c0-10d5-4ca0-8cbb-585dee61c94e", "device": "/dev/sdb"}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:09:05.441711 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-bff2528a-5991-4f1d-8e92-b6e15e8701ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:05.448005 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bff2528a-5991-4f1d-8e92-b6e15e8701ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:05.448455 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bff2528a-5991-4f1d-8e92-b6e15e8701ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:05.453860 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bff2528a-5991-4f1d-8e92-b6e15e8701ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:05.453860 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bff2528a-5991-4f1d-8e92-b6e15e8701ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:06.159861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-bff2528a-5991-4f1d-8e92-b6e15e8701ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "POST /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.723498 Jun 10 13:09:06.160620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 293/613] 10.222.0.25 () {60 vars in 1322 bytes} [Fri Jun 10 13:09:05 2022] POST /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments => generated 194 bytes in 726 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:09:20.793990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-923d4916-4956-4198-8474-c3a836e60789 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:20.794528 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-923d4916-4956-4198-8474-c3a836e60789 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:20.797018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=73 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:20.797661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=11 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:20.802969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-923d4916-4956-4198-8474-c3a836e60789 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:20.803414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-923d4916-4956-4198-8474-c3a836e60789 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:20.804170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-923d4916-4956-4198-8474-c3a836e60789 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:20.804601 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-923d4916-4956-4198-8474-c3a836e60789 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:20.834491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-923d4916-4956-4198-8474-c3a836e60789 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments/92fca8f5-5073-4d37-8ddf-ad6b329c0486" status: 200 len: 194 microversion: 2.20 time: 0.044523 Jun 10 13:09:20.835067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 321/614] 10.222.0.25 () {60 vars in 1453 bytes} [Fri Jun 10 13:09:20 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments/92fca8f5-5073-4d37-8ddf-ad6b329c0486 => generated 194 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:20.895099 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=41 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:20.896136 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:20.900561 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-022c8514-f5a7-4d4c-8acb-05e9dc3b8943 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:20.901167 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-022c8514-f5a7-4d4c-8acb-05e9dc3b8943 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:20.907484 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-022c8514-f5a7-4d4c-8acb-05e9dc3b8943 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:20.918721 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-022c8514-f5a7-4d4c-8acb-05e9dc3b8943 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:20.918721 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-022c8514-f5a7-4d4c-8acb-05e9dc3b8943 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:20.918721 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-022c8514-f5a7-4d4c-8acb-05e9dc3b8943 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:20.989952 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-022c8514-f5a7-4d4c-8acb-05e9dc3b8943 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lazy-loading 'services' on Instance uuid db56898a-dff9-4cad-bf8d-f84b53247fc3 {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:09:21.251439 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-022c8514-f5a7-4d4c-8acb-05e9dc3b8943 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "DELETE /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments/92fca8f5-5073-4d37-8ddf-ad6b329c0486" status: 202 len: 0 microversion: 2.20 time: 0.357752 Jun 10 13:09:21.252033 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 294/615] 10.222.0.25 () {60 vars in 1456 bytes} [Fri Jun 10 13:09:20 2022] DELETE /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments/92fca8f5-5073-4d37-8ddf-ad6b329c0486 => generated 0 bytes in 359 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jun 10 13:09:21.260782 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dcfe42a1-902a-4833-a1c4-b057f73395d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:21.261733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dcfe42a1-902a-4833-a1c4-b057f73395d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:21.268323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcfe42a1-902a-4833-a1c4-b057f73395d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:21.268323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcfe42a1-902a-4833-a1c4-b057f73395d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:21.269226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcfe42a1-902a-4833-a1c4-b057f73395d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:21.269659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcfe42a1-902a-4833-a1c4-b057f73395d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:21.301247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dcfe42a1-902a-4833-a1c4-b057f73395d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044615 Jun 10 13:09:21.301945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 322/616] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:21 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:22.315142 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2c9d3c29-0b75-407e-a9ca-c901e4813424 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:22.316104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2c9d3c29-0b75-407e-a9ca-c901e4813424 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:22.322377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c9d3c29-0b75-407e-a9ca-c901e4813424 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:22.322833 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c9d3c29-0b75-407e-a9ca-c901e4813424 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:22.323595 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c9d3c29-0b75-407e-a9ca-c901e4813424 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:22.324029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c9d3c29-0b75-407e-a9ca-c901e4813424 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:22.355342 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2c9d3c29-0b75-407e-a9ca-c901e4813424 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045055 Jun 10 13:09:22.355897 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 295/617] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:22 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:23.368614 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-76f16e5a-d8af-47f7-897c-096716ef9d73 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:23.369615 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-76f16e5a-d8af-47f7-897c-096716ef9d73 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:23.376131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76f16e5a-d8af-47f7-897c-096716ef9d73 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:23.376586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76f16e5a-d8af-47f7-897c-096716ef9d73 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:23.377382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76f16e5a-d8af-47f7-897c-096716ef9d73 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:23.377823 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76f16e5a-d8af-47f7-897c-096716ef9d73 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:23.410940 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-76f16e5a-d8af-47f7-897c-096716ef9d73 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047088 Jun 10 13:09:23.412156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 323/618] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:23 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:24.422793 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8daf83ee-118a-4bf1-819c-c9ccf1109e55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:24.423699 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8daf83ee-118a-4bf1-819c-c9ccf1109e55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:24.430259 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8daf83ee-118a-4bf1-819c-c9ccf1109e55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:24.430706 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8daf83ee-118a-4bf1-819c-c9ccf1109e55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:24.431459 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8daf83ee-118a-4bf1-819c-c9ccf1109e55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:24.431889 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8daf83ee-118a-4bf1-819c-c9ccf1109e55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:24.462085 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8daf83ee-118a-4bf1-819c-c9ccf1109e55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043086 Jun 10 13:09:24.462830 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 296/619] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:24 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:25.473831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fbe71b3c-6840-4c10-aa81-53bbdb23ecb7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:25.474740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fbe71b3c-6840-4c10-aa81-53bbdb23ecb7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:25.481207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fbe71b3c-6840-4c10-aa81-53bbdb23ecb7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:25.481650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fbe71b3c-6840-4c10-aa81-53bbdb23ecb7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:25.482407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fbe71b3c-6840-4c10-aa81-53bbdb23ecb7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:25.482839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fbe71b3c-6840-4c10-aa81-53bbdb23ecb7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:25.512779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fbe71b3c-6840-4c10-aa81-53bbdb23ecb7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042889 Jun 10 13:09:25.513333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 324/620] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:25 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:26.524878 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-59261e2b-4682-4d0a-9053-34c7e423237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:26.525785 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-59261e2b-4682-4d0a-9053-34c7e423237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:26.532224 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59261e2b-4682-4d0a-9053-34c7e423237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:26.532659 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59261e2b-4682-4d0a-9053-34c7e423237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:26.533441 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59261e2b-4682-4d0a-9053-34c7e423237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:26.533869 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59261e2b-4682-4d0a-9053-34c7e423237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:26.562071 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-59261e2b-4682-4d0a-9053-34c7e423237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041758 Jun 10 13:09:26.562609 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 297/621] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:26 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:27.573057 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-87fc954d-0354-4308-85ca-4a82390f814c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:27.573917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-87fc954d-0354-4308-85ca-4a82390f814c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:27.580337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87fc954d-0354-4308-85ca-4a82390f814c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:27.580811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87fc954d-0354-4308-85ca-4a82390f814c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:27.581570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87fc954d-0354-4308-85ca-4a82390f814c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:27.582002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87fc954d-0354-4308-85ca-4a82390f814c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:27.610160 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-87fc954d-0354-4308-85ca-4a82390f814c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041226 Jun 10 13:09:27.610698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 325/622] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:27 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:28.631085 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-25a6316a-9a4b-46a6-ba08-b1ed2800b4fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:28.631085 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-25a6316a-9a4b-46a6-ba08-b1ed2800b4fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:28.631085 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-25a6316a-9a4b-46a6-ba08-b1ed2800b4fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:28.631085 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-25a6316a-9a4b-46a6-ba08-b1ed2800b4fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:28.631905 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-25a6316a-9a4b-46a6-ba08-b1ed2800b4fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:28.631905 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-25a6316a-9a4b-46a6-ba08-b1ed2800b4fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:28.659766 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-25a6316a-9a4b-46a6-ba08-b1ed2800b4fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041641 Jun 10 13:09:28.663842 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 298/623] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:28 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:29.671577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8c82a7a3-22fe-4896-95dd-8bfa92487d55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:29.672394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8c82a7a3-22fe-4896-95dd-8bfa92487d55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:29.679501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8c82a7a3-22fe-4896-95dd-8bfa92487d55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:29.679944 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8c82a7a3-22fe-4896-95dd-8bfa92487d55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:29.680701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8c82a7a3-22fe-4896-95dd-8bfa92487d55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:29.681162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8c82a7a3-22fe-4896-95dd-8bfa92487d55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:29.708753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8c82a7a3-22fe-4896-95dd-8bfa92487d55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041235 Jun 10 13:09:29.709313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 326/624] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:29 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:30.720341 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9995e190-c989-4cc4-a0b9-9d3b0799cb3e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:30.721283 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9995e190-c989-4cc4-a0b9-9d3b0799cb3e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:30.727772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9995e190-c989-4cc4-a0b9-9d3b0799cb3e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:30.728217 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9995e190-c989-4cc4-a0b9-9d3b0799cb3e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:30.729011 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9995e190-c989-4cc4-a0b9-9d3b0799cb3e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:30.729443 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9995e190-c989-4cc4-a0b9-9d3b0799cb3e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:30.758207 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9995e190-c989-4cc4-a0b9-9d3b0799cb3e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041835 Jun 10 13:09:30.759326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 299/625] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:30 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:31.770763 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ee4fde1b-e9ea-428d-a56c-dccfe169a766 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:31.771659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ee4fde1b-e9ea-428d-a56c-dccfe169a766 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:31.778271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee4fde1b-e9ea-428d-a56c-dccfe169a766 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:31.778720 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee4fde1b-e9ea-428d-a56c-dccfe169a766 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:31.784296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee4fde1b-e9ea-428d-a56c-dccfe169a766 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:31.784296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee4fde1b-e9ea-428d-a56c-dccfe169a766 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:31.809068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ee4fde1b-e9ea-428d-a56c-dccfe169a766 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042434 Jun 10 13:09:31.809625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 327/626] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:31 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:32.819760 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c05e9871-058f-493f-8289-729d433386c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:32.820675 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c05e9871-058f-493f-8289-729d433386c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:32.827416 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c05e9871-058f-493f-8289-729d433386c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:32.827861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c05e9871-058f-493f-8289-729d433386c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:32.828627 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c05e9871-058f-493f-8289-729d433386c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:32.829091 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c05e9871-058f-493f-8289-729d433386c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:32.857279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c05e9871-058f-493f-8289-729d433386c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041501 Jun 10 13:09:32.857824 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 300/627] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:32 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:33.869860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4b8c2946-5185-44a4-9d7c-457ca1ab350d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:33.870802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4b8c2946-5185-44a4-9d7c-457ca1ab350d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:33.877323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b8c2946-5185-44a4-9d7c-457ca1ab350d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:33.877768 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b8c2946-5185-44a4-9d7c-457ca1ab350d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:33.878518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b8c2946-5185-44a4-9d7c-457ca1ab350d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:33.878981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b8c2946-5185-44a4-9d7c-457ca1ab350d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:33.907523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4b8c2946-5185-44a4-9d7c-457ca1ab350d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042146 Jun 10 13:09:33.908074 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 328/628] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:33 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:34.919107 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3be24f07-b00b-4ae4-b970-a1a04aa36f67 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:34.920052 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3be24f07-b00b-4ae4-b970-a1a04aa36f67 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:34.926639 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3be24f07-b00b-4ae4-b970-a1a04aa36f67 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:34.927090 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3be24f07-b00b-4ae4-b970-a1a04aa36f67 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:34.927850 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3be24f07-b00b-4ae4-b970-a1a04aa36f67 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:34.928288 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3be24f07-b00b-4ae4-b970-a1a04aa36f67 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:34.968785 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3be24f07-b00b-4ae4-b970-a1a04aa36f67 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053581 Jun 10 13:09:34.969364 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 301/629] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:34 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:09:35.981139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9b6df093-9699-4f6d-884f-03d8c4f99cb0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:35.982315 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9b6df093-9699-4f6d-884f-03d8c4f99cb0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:35.990397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9b6df093-9699-4f6d-884f-03d8c4f99cb0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:35.990838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9b6df093-9699-4f6d-884f-03d8c4f99cb0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:35.991618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9b6df093-9699-4f6d-884f-03d8c4f99cb0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:35.992047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9b6df093-9699-4f6d-884f-03d8c4f99cb0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:36.020350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9b6df093-9699-4f6d-884f-03d8c4f99cb0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043869 Jun 10 13:09:36.020937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 329/630] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:35 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:37.032508 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-73b51306-fbd1-4879-b41d-601dbf558eb1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:37.033381 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-73b51306-fbd1-4879-b41d-601dbf558eb1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:37.039809 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-73b51306-fbd1-4879-b41d-601dbf558eb1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:37.040248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-73b51306-fbd1-4879-b41d-601dbf558eb1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:37.041024 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-73b51306-fbd1-4879-b41d-601dbf558eb1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:37.041459 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-73b51306-fbd1-4879-b41d-601dbf558eb1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:37.069693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-73b51306-fbd1-4879-b41d-601dbf558eb1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041216 Jun 10 13:09:37.070232 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 302/631] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:37 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:38.081607 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-550882f4-625c-490a-be46-9e1e727bd5b8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:38.082545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-550882f4-625c-490a-be46-9e1e727bd5b8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:38.090417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-550882f4-625c-490a-be46-9e1e727bd5b8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:38.090989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-550882f4-625c-490a-be46-9e1e727bd5b8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:38.092307 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-550882f4-625c-490a-be46-9e1e727bd5b8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:38.093113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-550882f4-625c-490a-be46-9e1e727bd5b8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:38.123379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-550882f4-625c-490a-be46-9e1e727bd5b8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045765 Jun 10 13:09:38.123938 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 330/632] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:38 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:39.135715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4b26ccc7-9f80-4760-b1f9-4d618a001f50 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:39.136626 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4b26ccc7-9f80-4760-b1f9-4d618a001f50 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:39.143159 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b26ccc7-9f80-4760-b1f9-4d618a001f50 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:39.143600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b26ccc7-9f80-4760-b1f9-4d618a001f50 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:39.144349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b26ccc7-9f80-4760-b1f9-4d618a001f50 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:39.144793 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b26ccc7-9f80-4760-b1f9-4d618a001f50 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:39.173131 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4b26ccc7-9f80-4760-b1f9-4d618a001f50 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041971 Jun 10 13:09:39.173672 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 303/633] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:39 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:09:40.184049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c3219282-a171-4429-b709-fbe920af3763 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:40.184983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c3219282-a171-4429-b709-fbe920af3763 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:40.191341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3219282-a171-4429-b709-fbe920af3763 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:40.191780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3219282-a171-4429-b709-fbe920af3763 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:40.192529 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3219282-a171-4429-b709-fbe920af3763 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:40.192991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3219282-a171-4429-b709-fbe920af3763 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:40.220439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c3219282-a171-4429-b709-fbe920af3763 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041004 Jun 10 13:09:40.221008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 331/634] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:40 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:41.232070 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9f93bc4c-c2d5-4636-8010-268219dbf097 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:41.233033 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9f93bc4c-c2d5-4636-8010-268219dbf097 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:41.239700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9f93bc4c-c2d5-4636-8010-268219dbf097 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:41.240142 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9f93bc4c-c2d5-4636-8010-268219dbf097 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:41.240935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9f93bc4c-c2d5-4636-8010-268219dbf097 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:41.241370 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9f93bc4c-c2d5-4636-8010-268219dbf097 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:41.270378 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9f93bc4c-c2d5-4636-8010-268219dbf097 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042809 Jun 10 13:09:41.270941 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 304/635] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:41 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:42.285072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9a702249-57ff-4958-b4e9-5c1895fb5004 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:42.286105 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9a702249-57ff-4958-b4e9-5c1895fb5004 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:42.293299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a702249-57ff-4958-b4e9-5c1895fb5004 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:42.293741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a702249-57ff-4958-b4e9-5c1895fb5004 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:42.294494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a702249-57ff-4958-b4e9-5c1895fb5004 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:42.294927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a702249-57ff-4958-b4e9-5c1895fb5004 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:42.325406 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9a702249-57ff-4958-b4e9-5c1895fb5004 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046681 Jun 10 13:09:42.325976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 332/636] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:42 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:43.090391 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6c3ebd29-4df8-4d91-adfb-6f86dc08c14e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:43.091318 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6c3ebd29-4df8-4d91-adfb-6f86dc08c14e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:43.097966 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c3ebd29-4df8-4d91-adfb-6f86dc08c14e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:43.098441 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c3ebd29-4df8-4d91-adfb-6f86dc08c14e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:43.099184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c3ebd29-4df8-4d91-adfb-6f86dc08c14e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:43.099614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6c3ebd29-4df8-4d91-adfb-6f86dc08c14e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:43.127559 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6c3ebd29-4df8-4d91-adfb-6f86dc08c14e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041682 Jun 10 13:09:43.128091 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 305/637] 10.222.0.25 () {58 vars in 1301 bytes} [Fri Jun 10 13:09:43 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jun 10 13:09:43.185397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-43e86381-c5ee-4c43-b573-9e35196f8208 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:43.185893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-43e86381-c5ee-4c43-b573-9e35196f8208 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:43.191957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-43e86381-c5ee-4c43-b573-9e35196f8208 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:43.196290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-43e86381-c5ee-4c43-b573-9e35196f8208 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:43.196290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-43e86381-c5ee-4c43-b573-9e35196f8208 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:43.196290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-43e86381-c5ee-4c43-b573-9e35196f8208 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:43.218858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-43e86381-c5ee-4c43-b573-9e35196f8208 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:09:43.338305 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-256ffad4-4336-41c0-8e7e-785356132713 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:43.339216 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-256ffad4-4336-41c0-8e7e-785356132713 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:43.345200 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-256ffad4-4336-41c0-8e7e-785356132713 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:43.345638 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-256ffad4-4336-41c0-8e7e-785356132713 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:43.346506 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-256ffad4-4336-41c0-8e7e-785356132713 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:43.346870 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-256ffad4-4336-41c0-8e7e-785356132713 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:43.374970 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-256ffad4-4336-41c0-8e7e-785356132713 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041290 Jun 10 13:09:43.375517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 306/638] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:43 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:09:43.480660 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-43e86381-c5ee-4c43-b573-9e35196f8208 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "DELETE /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 204 len: 0 microversion: 2.1 time: 0.298758 Jun 10 13:09:43.481284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 333/639] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:09:43 2022] DELETE /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jun 10 13:09:43.489525 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cb6455b0-5953-40ae-b293-9013139283dd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:43.494716 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cb6455b0-5953-40ae-b293-9013139283dd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:43.496788 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cb6455b0-5953-40ae-b293-9013139283dd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:43.497463 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cb6455b0-5953-40ae-b293-9013139283dd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:43.498323 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cb6455b0-5953-40ae-b293-9013139283dd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:43.498771 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cb6455b0-5953-40ae-b293-9013139283dd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:43.634914 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cb6455b0-5953-40ae-b293-9013139283dd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:43.639018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ebaedba6-ea7b-4da2-83d7-557ad4360a0a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "84a30ba8-3eb4-4c2c-9a53-6d45a7a7dd49"}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:09:43.640387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ebaedba6-ea7b-4da2-83d7-557ad4360a0a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:43.642449 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cb6455b0-5953-40ae-b293-9013139283dd tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.156985 Jun 10 13:09:43.643145 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 307/640] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:43 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:09:43.646629 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebaedba6-ea7b-4da2-83d7-557ad4360a0a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:43.647078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebaedba6-ea7b-4da2-83d7-557ad4360a0a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:43.647882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebaedba6-ea7b-4da2-83d7-557ad4360a0a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:43.648315 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebaedba6-ea7b-4da2-83d7-557ad4360a0a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:44.386490 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d8ccc4a4-7a71-4fd2-91a6-b0a579cdc71f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:44.387393 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d8ccc4a4-7a71-4fd2-91a6-b0a579cdc71f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:44.393789 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d8ccc4a4-7a71-4fd2-91a6-b0a579cdc71f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:44.394229 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d8ccc4a4-7a71-4fd2-91a6-b0a579cdc71f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:44.394974 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d8ccc4a4-7a71-4fd2-91a6-b0a579cdc71f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:44.395400 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d8ccc4a4-7a71-4fd2-91a6-b0a579cdc71f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:44.424085 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d8ccc4a4-7a71-4fd2-91a6-b0a579cdc71f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041529 Jun 10 13:09:44.424617 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 308/641] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:44 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:09:44.659232 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-10e334c6-ad19-4046-8d05-341a24a3aa8d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:44.659683 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-10e334c6-ad19-4046-8d05-341a24a3aa8d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:44.665841 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-10e334c6-ad19-4046-8d05-341a24a3aa8d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:44.666294 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-10e334c6-ad19-4046-8d05-341a24a3aa8d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:44.667127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-10e334c6-ad19-4046-8d05-341a24a3aa8d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:44.667568 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-10e334c6-ad19-4046-8d05-341a24a3aa8d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:44.794488 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-10e334c6-ad19-4046-8d05-341a24a3aa8d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:44.797990 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:44.802347 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-10e334c6-ad19-4046-8d05-341a24a3aa8d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.147002 Jun 10 13:09:44.802916 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 309/642] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:44 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:09:45.436282 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cebd6876-6614-4652-965b-bd755e168712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:45.437220 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cebd6876-6614-4652-965b-bd755e168712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:45.444306 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cebd6876-6614-4652-965b-bd755e168712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:45.444752 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cebd6876-6614-4652-965b-bd755e168712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:45.445533 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cebd6876-6614-4652-965b-bd755e168712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:45.445966 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cebd6876-6614-4652-965b-bd755e168712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:45.474201 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cebd6876-6614-4652-965b-bd755e168712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041863 Jun 10 13:09:45.474752 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 310/643] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:45 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:09:45.823326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4e31dd3c-c661-42de-b121-120a1f75458c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:45.823937 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4e31dd3c-c661-42de-b121-120a1f75458c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:45.829868 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4e31dd3c-c661-42de-b121-120a1f75458c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:45.830312 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4e31dd3c-c661-42de-b121-120a1f75458c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:45.831065 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4e31dd3c-c661-42de-b121-120a1f75458c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:45.831497 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4e31dd3c-c661-42de-b121-120a1f75458c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:45.955300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4e31dd3c-c661-42de-b121-120a1f75458c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:45.963166 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4e31dd3c-c661-42de-b121-120a1f75458c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.143755 Jun 10 13:09:45.963730 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 311/644] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:45 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:09:46.485692 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c7cd0c84-41ae-4e26-ac21-893b356fb7a1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:46.486595 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c7cd0c84-41ae-4e26-ac21-893b356fb7a1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:46.493778 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c7cd0c84-41ae-4e26-ac21-893b356fb7a1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:46.494234 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c7cd0c84-41ae-4e26-ac21-893b356fb7a1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:46.494967 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c7cd0c84-41ae-4e26-ac21-893b356fb7a1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:46.495397 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c7cd0c84-41ae-4e26-ac21-893b356fb7a1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:46.523514 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c7cd0c84-41ae-4e26-ac21-893b356fb7a1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041796 Jun 10 13:09:46.524175 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 312/645] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:46 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:09:46.979891 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7e5c5cf9-b6d6-46df-ac3d-d7a63932f966 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:46.980495 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7e5c5cf9-b6d6-46df-ac3d-d7a63932f966 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:46.986442 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7e5c5cf9-b6d6-46df-ac3d-d7a63932f966 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:46.986884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7e5c5cf9-b6d6-46df-ac3d-d7a63932f966 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:46.987884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7e5c5cf9-b6d6-46df-ac3d-d7a63932f966 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:46.988313 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7e5c5cf9-b6d6-46df-ac3d-d7a63932f966 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:47.115760 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7e5c5cf9-b6d6-46df-ac3d-d7a63932f966 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:47.123168 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7e5c5cf9-b6d6-46df-ac3d-d7a63932f966 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.147633 Jun 10 13:09:47.123743 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 313/646] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:46 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:09:47.534047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cb5fa1ac-4f4b-4343-82f2-c33412185f7a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:47.534964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cb5fa1ac-4f4b-4343-82f2-c33412185f7a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:47.542667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cb5fa1ac-4f4b-4343-82f2-c33412185f7a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:47.544538 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cb5fa1ac-4f4b-4343-82f2-c33412185f7a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:47.546367 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cb5fa1ac-4f4b-4343-82f2-c33412185f7a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:47.546811 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cb5fa1ac-4f4b-4343-82f2-c33412185f7a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:47.576334 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cb5fa1ac-4f4b-4343-82f2-c33412185f7a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046215 Jun 10 13:09:47.576920 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 314/647] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:47 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:09:48.140203 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-05c7a734-0ec0-4036-8310-7046da297ee4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:48.140725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-05c7a734-0ec0-4036-8310-7046da297ee4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:48.146995 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-05c7a734-0ec0-4036-8310-7046da297ee4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:48.147435 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-05c7a734-0ec0-4036-8310-7046da297ee4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:48.148188 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-05c7a734-0ec0-4036-8310-7046da297ee4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:48.148615 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-05c7a734-0ec0-4036-8310-7046da297ee4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:48.275914 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-05c7a734-0ec0-4036-8310-7046da297ee4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:48.283232 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-05c7a734-0ec0-4036-8310-7046da297ee4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.146934 Jun 10 13:09:48.283789 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 315/648] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:48 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:09:48.588964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-809bd55c-8f63-45c2-be2c-01b4de6dd3e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:48.589874 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-809bd55c-8f63-45c2-be2c-01b4de6dd3e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:48.596403 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-809bd55c-8f63-45c2-be2c-01b4de6dd3e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:48.596873 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-809bd55c-8f63-45c2-be2c-01b4de6dd3e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:48.597628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-809bd55c-8f63-45c2-be2c-01b4de6dd3e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:48.598054 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-809bd55c-8f63-45c2-be2c-01b4de6dd3e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:48.627816 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-809bd55c-8f63-45c2-be2c-01b4de6dd3e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043746 Jun 10 13:09:48.628365 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 316/649] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:48 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:09:49.301002 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-41a8f94a-841e-4165-a34b-04a043bd53e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:49.301632 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-41a8f94a-841e-4165-a34b-04a043bd53e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:49.307661 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-41a8f94a-841e-4165-a34b-04a043bd53e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:49.308104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-41a8f94a-841e-4165-a34b-04a043bd53e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:49.308875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-41a8f94a-841e-4165-a34b-04a043bd53e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:49.309299 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-41a8f94a-841e-4165-a34b-04a043bd53e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:49.434929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-41a8f94a-841e-4165-a34b-04a043bd53e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:49.442395 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-41a8f94a-841e-4165-a34b-04a043bd53e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.145291 Jun 10 13:09:49.442946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 317/650] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:49 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:09:49.639410 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b3099471-a1e4-496b-96c2-a29bf54d1fc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:49.640316 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b3099471-a1e4-496b-96c2-a29bf54d1fc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:49.646775 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b3099471-a1e4-496b-96c2-a29bf54d1fc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:49.647221 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b3099471-a1e4-496b-96c2-a29bf54d1fc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:49.647976 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b3099471-a1e4-496b-96c2-a29bf54d1fc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:49.648409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b3099471-a1e4-496b-96c2-a29bf54d1fc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:49.676161 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b3099471-a1e4-496b-96c2-a29bf54d1fc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040647 Jun 10 13:09:49.676644 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 318/651] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:49 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:09:50.463880 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-40b882c9-4d7c-4d75-9812-5cebb4b69cbc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:50.463880 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-40b882c9-4d7c-4d75-9812-5cebb4b69cbc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:50.467381 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-40b882c9-4d7c-4d75-9812-5cebb4b69cbc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:50.467821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-40b882c9-4d7c-4d75-9812-5cebb4b69cbc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:50.468570 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-40b882c9-4d7c-4d75-9812-5cebb4b69cbc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:50.469023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-40b882c9-4d7c-4d75-9812-5cebb4b69cbc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:50.595581 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-40b882c9-4d7c-4d75-9812-5cebb4b69cbc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:50.602908 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-40b882c9-4d7c-4d75-9812-5cebb4b69cbc tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.147022 Jun 10 13:09:50.603493 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 319/652] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:50 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:09:50.686870 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-66eb1c05-d5f4-4aab-83d6-78f5fa1ea835 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:50.687765 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-66eb1c05-d5f4-4aab-83d6-78f5fa1ea835 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:50.693866 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66eb1c05-d5f4-4aab-83d6-78f5fa1ea835 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:50.694307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66eb1c05-d5f4-4aab-83d6-78f5fa1ea835 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:50.695051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66eb1c05-d5f4-4aab-83d6-78f5fa1ea835 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:50.695478 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66eb1c05-d5f4-4aab-83d6-78f5fa1ea835 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:50.723908 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-66eb1c05-d5f4-4aab-83d6-78f5fa1ea835 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040704 Jun 10 13:09:50.724439 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 320/653] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:50 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:09:51.621307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-baafe285-5fbf-4c65-9e3a-92252419fe94 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:51.622054 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-baafe285-5fbf-4c65-9e3a-92252419fe94 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:51.623945 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=55 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:51.628676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-baafe285-5fbf-4c65-9e3a-92252419fe94 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:51.629154 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-baafe285-5fbf-4c65-9e3a-92252419fe94 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:51.629950 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-baafe285-5fbf-4c65-9e3a-92252419fe94 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:51.630384 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-baafe285-5fbf-4c65-9e3a-92252419fe94 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:51.767252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-baafe285-5fbf-4c65-9e3a-92252419fe94 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:51.775210 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-baafe285-5fbf-4c65-9e3a-92252419fe94 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.158728 Jun 10 13:09:51.775769 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 321/654] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:51 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:09:51.780287 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9fc77959-a842-4e2e-8296-51922207b0fd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:51.781207 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9fc77959-a842-4e2e-8296-51922207b0fd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:51.786845 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9fc77959-a842-4e2e-8296-51922207b0fd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:51.788875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9fc77959-a842-4e2e-8296-51922207b0fd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:51.788875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9fc77959-a842-4e2e-8296-51922207b0fd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:51.788875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9fc77959-a842-4e2e-8296-51922207b0fd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:51.816907 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9fc77959-a842-4e2e-8296-51922207b0fd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040128 Jun 10 13:09:51.817443 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 322/655] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:51 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:52.792796 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-607b236e-8946-45c5-a446-cfd1bb4f828d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:52.793446 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-607b236e-8946-45c5-a446-cfd1bb4f828d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:52.799773 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-607b236e-8946-45c5-a446-cfd1bb4f828d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:52.800212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-607b236e-8946-45c5-a446-cfd1bb4f828d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:52.800991 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-607b236e-8946-45c5-a446-cfd1bb4f828d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:52.801417 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-607b236e-8946-45c5-a446-cfd1bb4f828d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:52.929231 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-607b236e-8946-45c5-a446-cfd1bb4f828d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:52.936664 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-607b236e-8946-45c5-a446-cfd1bb4f828d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.148266 Jun 10 13:09:52.937262 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 323/656] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:52 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:09:52.941869 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-dc8e29da-cd5b-40b4-8c12-7c3c8b3ca7a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:52.942764 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-dc8e29da-cd5b-40b4-8c12-7c3c8b3ca7a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:52.948963 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dc8e29da-cd5b-40b4-8c12-7c3c8b3ca7a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:52.949439 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dc8e29da-cd5b-40b4-8c12-7c3c8b3ca7a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:52.950199 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dc8e29da-cd5b-40b4-8c12-7c3c8b3ca7a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:52.950632 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dc8e29da-cd5b-40b4-8c12-7c3c8b3ca7a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:52.978750 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-dc8e29da-cd5b-40b4-8c12-7c3c8b3ca7a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040432 Jun 10 13:09:52.979294 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 324/657] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:52 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:53.644091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:53.719033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:53.954074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-197624c7-f117-422c-81b5-a9c9a4d59668 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:53.954686 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-197624c7-f117-422c-81b5-a9c9a4d59668 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:53.961727 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-197624c7-f117-422c-81b5-a9c9a4d59668 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:53.962187 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-197624c7-f117-422c-81b5-a9c9a4d59668 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:53.962941 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-197624c7-f117-422c-81b5-a9c9a4d59668 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:53.963370 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-197624c7-f117-422c-81b5-a9c9a4d59668 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:54.093300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-197624c7-f117-422c-81b5-a9c9a4d59668 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:54.100607 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-197624c7-f117-422c-81b5-a9c9a4d59668 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.150291 Jun 10 13:09:54.101217 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 325/658] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:53 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:09:54.106361 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-00deabc0-4b66-4450-ae6c-4c4860250430 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:54.107252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-00deabc0-4b66-4450-ae6c-4c4860250430 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:54.113033 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-00deabc0-4b66-4450-ae6c-4c4860250430 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:54.113482 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-00deabc0-4b66-4450-ae6c-4c4860250430 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:54.114228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-00deabc0-4b66-4450-ae6c-4c4860250430 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:54.114655 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-00deabc0-4b66-4450-ae6c-4c4860250430 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:54.131378 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:54.143347 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-00deabc0-4b66-4450-ae6c-4c4860250430 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041175 Jun 10 13:09:54.150532 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 326/659] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:54 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:55.119361 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a8598448-e22b-48c3-8847-79de09807b1d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:55.119993 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a8598448-e22b-48c3-8847-79de09807b1d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:55.126550 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a8598448-e22b-48c3-8847-79de09807b1d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:55.127000 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a8598448-e22b-48c3-8847-79de09807b1d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:55.127758 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a8598448-e22b-48c3-8847-79de09807b1d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:55.128190 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a8598448-e22b-48c3-8847-79de09807b1d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:55.252046 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a8598448-e22b-48c3-8847-79de09807b1d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:55.259537 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a8598448-e22b-48c3-8847-79de09807b1d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.144895 Jun 10 13:09:55.260104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 327/660] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:55 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:09:55.264675 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6fda5e9d-6f02-48be-8ed9-bbfcedba235b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:55.265601 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6fda5e9d-6f02-48be-8ed9-bbfcedba235b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:55.271200 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6fda5e9d-6f02-48be-8ed9-bbfcedba235b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:55.271640 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6fda5e9d-6f02-48be-8ed9-bbfcedba235b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:55.272422 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6fda5e9d-6f02-48be-8ed9-bbfcedba235b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:55.272890 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6fda5e9d-6f02-48be-8ed9-bbfcedba235b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:55.301144 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6fda5e9d-6f02-48be-8ed9-bbfcedba235b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040026 Jun 10 13:09:55.301693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 328/661] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:55 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:56.276710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3d197946-a158-443d-b4df-07143ea07f70 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:56.277408 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3d197946-a158-443d-b4df-07143ea07f70 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:56.283758 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d197946-a158-443d-b4df-07143ea07f70 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:56.284202 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d197946-a158-443d-b4df-07143ea07f70 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:56.285011 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d197946-a158-443d-b4df-07143ea07f70 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:56.285446 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d197946-a158-443d-b4df-07143ea07f70 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:56.413352 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3d197946-a158-443d-b4df-07143ea07f70 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:56.421407 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3d197946-a158-443d-b4df-07143ea07f70 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.149091 Jun 10 13:09:56.421971 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 329/662] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:56 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:09:56.426425 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-391f2d78-3a6f-4e28-98f6-bac2df002d57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:56.427317 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-391f2d78-3a6f-4e28-98f6-bac2df002d57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:56.432879 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-391f2d78-3a6f-4e28-98f6-bac2df002d57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:56.433326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-391f2d78-3a6f-4e28-98f6-bac2df002d57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:56.434076 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-391f2d78-3a6f-4e28-98f6-bac2df002d57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:56.434507 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-391f2d78-3a6f-4e28-98f6-bac2df002d57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:56.461836 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-391f2d78-3a6f-4e28-98f6-bac2df002d57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.038894 Jun 10 13:09:56.462884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 330/663] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:56 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:57.438737 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cc9f4513-311d-4749-bfa1-e32b6c9a6f2d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:57.439373 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cc9f4513-311d-4749-bfa1-e32b6c9a6f2d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:57.445743 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc9f4513-311d-4749-bfa1-e32b6c9a6f2d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:57.446191 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc9f4513-311d-4749-bfa1-e32b6c9a6f2d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:57.446943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc9f4513-311d-4749-bfa1-e32b6c9a6f2d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:57.447376 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc9f4513-311d-4749-bfa1-e32b6c9a6f2d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:57.571560 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cc9f4513-311d-4749-bfa1-e32b6c9a6f2d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:57.578923 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cc9f4513-311d-4749-bfa1-e32b6c9a6f2d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.144085 Jun 10 13:09:57.579476 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 331/664] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:57 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:09:57.583873 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5537097c-733a-4e29-b6e6-a34c3bd96e60 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:57.584756 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5537097c-733a-4e29-b6e6-a34c3bd96e60 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:57.590667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5537097c-733a-4e29-b6e6-a34c3bd96e60 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:57.591054 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5537097c-733a-4e29-b6e6-a34c3bd96e60 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:57.591840 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5537097c-733a-4e29-b6e6-a34c3bd96e60 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:57.592270 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5537097c-733a-4e29-b6e6-a34c3bd96e60 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:57.624470 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5537097c-733a-4e29-b6e6-a34c3bd96e60 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043994 Jun 10 13:09:57.625058 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 332/665] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:57 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:58.595598 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d70891dc-3f07-4031-ac14-78acb5b34fa9 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:58.596217 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d70891dc-3f07-4031-ac14-78acb5b34fa9 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:58.602652 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d70891dc-3f07-4031-ac14-78acb5b34fa9 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:58.603099 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d70891dc-3f07-4031-ac14-78acb5b34fa9 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:58.603853 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d70891dc-3f07-4031-ac14-78acb5b34fa9 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:58.604285 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d70891dc-3f07-4031-ac14-78acb5b34fa9 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:58.733299 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d70891dc-3f07-4031-ac14-78acb5b34fa9 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:58.741314 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d70891dc-3f07-4031-ac14-78acb5b34fa9 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.149630 Jun 10 13:09:58.741886 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 333/666] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:58 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:09:58.747067 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-26dc7416-a4e6-4f93-a113-c21966184f05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:58.747966 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-26dc7416-a4e6-4f93-a113-c21966184f05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:58.753570 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26dc7416-a4e6-4f93-a113-c21966184f05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:58.754015 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26dc7416-a4e6-4f93-a113-c21966184f05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:58.754800 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26dc7416-a4e6-4f93-a113-c21966184f05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:58.755230 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26dc7416-a4e6-4f93-a113-c21966184f05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:58.783333 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-26dc7416-a4e6-4f93-a113-c21966184f05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040445 Jun 10 13:09:58.783894 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 334/667] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:58 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:09:59.758147 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1cdfa626-0247-40f8-951a-2aec2be527e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:59.758764 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1cdfa626-0247-40f8-951a-2aec2be527e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:59.765787 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1cdfa626-0247-40f8-951a-2aec2be527e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:59.766233 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1cdfa626-0247-40f8-951a-2aec2be527e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:59.766989 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1cdfa626-0247-40f8-951a-2aec2be527e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:59.767423 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1cdfa626-0247-40f8-951a-2aec2be527e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:59.893919 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1cdfa626-0247-40f8-951a-2aec2be527e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:09:59.901891 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1cdfa626-0247-40f8-951a-2aec2be527e4 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.147469 Jun 10 13:09:59.908869 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 335/668] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:09:59 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:09:59.908869 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a1e15440-fed6-4e23-ae6a-725f02a06e37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:09:59.908869 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a1e15440-fed6-4e23-ae6a-725f02a06e37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:09:59.913648 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a1e15440-fed6-4e23-ae6a-725f02a06e37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:59.914090 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a1e15440-fed6-4e23-ae6a-725f02a06e37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:59.914844 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a1e15440-fed6-4e23-ae6a-725f02a06e37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:09:59.915274 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a1e15440-fed6-4e23-ae6a-725f02a06e37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:09:59.943346 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a1e15440-fed6-4e23-ae6a-725f02a06e37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.039934 Jun 10 13:09:59.943878 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 336/669] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:09:59 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:00.919470 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-99e2187e-6956-4052-9bed-6dc9fa891eea tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:00.920102 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-99e2187e-6956-4052-9bed-6dc9fa891eea tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:00.926641 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-99e2187e-6956-4052-9bed-6dc9fa891eea tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:00.927089 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-99e2187e-6956-4052-9bed-6dc9fa891eea tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:00.927852 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-99e2187e-6956-4052-9bed-6dc9fa891eea tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:00.928287 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-99e2187e-6956-4052-9bed-6dc9fa891eea tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:01.051961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-99e2187e-6956-4052-9bed-6dc9fa891eea tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:01.059499 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-99e2187e-6956-4052-9bed-6dc9fa891eea tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.144442 Jun 10 13:10:01.060063 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 337/670] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:00 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:10:01.064556 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-66497f16-b4f0-47f1-92a9-01bf7648210e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:01.065499 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-66497f16-b4f0-47f1-92a9-01bf7648210e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:01.071192 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66497f16-b4f0-47f1-92a9-01bf7648210e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:01.072145 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66497f16-b4f0-47f1-92a9-01bf7648210e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:01.072923 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66497f16-b4f0-47f1-92a9-01bf7648210e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:01.073356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66497f16-b4f0-47f1-92a9-01bf7648210e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:01.101999 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-66497f16-b4f0-47f1-92a9-01bf7648210e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040931 Jun 10 13:10:01.103291 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 338/671] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:01 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:02.077087 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7d331986-ab3d-4984-a6e5-e976b9dde174 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:02.077688 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7d331986-ab3d-4984-a6e5-e976b9dde174 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:02.084074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7d331986-ab3d-4984-a6e5-e976b9dde174 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:02.084519 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7d331986-ab3d-4984-a6e5-e976b9dde174 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:02.085317 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7d331986-ab3d-4984-a6e5-e976b9dde174 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:02.085752 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7d331986-ab3d-4984-a6e5-e976b9dde174 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:02.216346 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7d331986-ab3d-4984-a6e5-e976b9dde174 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:02.223843 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7d331986-ab3d-4984-a6e5-e976b9dde174 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.150713 Jun 10 13:10:02.224405 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 339/672] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:02 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:10:02.229435 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-78d93a03-7109-4239-a50c-86560cb2971d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:02.230318 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-78d93a03-7109-4239-a50c-86560cb2971d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:02.235703 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-78d93a03-7109-4239-a50c-86560cb2971d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:02.236144 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-78d93a03-7109-4239-a50c-86560cb2971d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:02.236934 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-78d93a03-7109-4239-a50c-86560cb2971d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:02.237366 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-78d93a03-7109-4239-a50c-86560cb2971d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:02.264810 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-78d93a03-7109-4239-a50c-86560cb2971d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.039341 Jun 10 13:10:02.265356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 340/673] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:02 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:03.241511 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-74971044-5377-456d-8fbc-23702df7a117 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:03.242105 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-74971044-5377-456d-8fbc-23702df7a117 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:03.248475 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-74971044-5377-456d-8fbc-23702df7a117 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:03.248962 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-74971044-5377-456d-8fbc-23702df7a117 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:03.249711 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-74971044-5377-456d-8fbc-23702df7a117 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:03.250138 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-74971044-5377-456d-8fbc-23702df7a117 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:03.527131 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-74971044-5377-456d-8fbc-23702df7a117 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:03.534529 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-74971044-5377-456d-8fbc-23702df7a117 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.297177 Jun 10 13:10:03.535105 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 341/674] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:03 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:10:03.539736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cacd8937-be9d-49b2-a90c-2ad42dee5d29 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:03.540635 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cacd8937-be9d-49b2-a90c-2ad42dee5d29 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:03.546151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cacd8937-be9d-49b2-a90c-2ad42dee5d29 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:03.546599 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cacd8937-be9d-49b2-a90c-2ad42dee5d29 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:03.547358 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cacd8937-be9d-49b2-a90c-2ad42dee5d29 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:03.547794 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cacd8937-be9d-49b2-a90c-2ad42dee5d29 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:03.575835 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cacd8937-be9d-49b2-a90c-2ad42dee5d29 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.039675 Jun 10 13:10:03.576378 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 342/675] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:03 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:04.551526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8b30f60d-7507-4f68-b823-ccbada249e87 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:04.552123 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8b30f60d-7507-4f68-b823-ccbada249e87 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:04.558671 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8b30f60d-7507-4f68-b823-ccbada249e87 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:04.559108 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8b30f60d-7507-4f68-b823-ccbada249e87 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:04.559854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8b30f60d-7507-4f68-b823-ccbada249e87 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:04.560281 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8b30f60d-7507-4f68-b823-ccbada249e87 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:04.564170 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:04.926779 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8b30f60d-7507-4f68-b823-ccbada249e87 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:04.934121 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8b30f60d-7507-4f68-b823-ccbada249e87 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.386404 Jun 10 13:10:04.934676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 343/676] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:04 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:10:04.939214 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c173e483-9d39-4c96-90bd-2ab711dce4b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:04.940102 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c173e483-9d39-4c96-90bd-2ab711dce4b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:04.945501 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c173e483-9d39-4c96-90bd-2ab711dce4b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:04.945938 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c173e483-9d39-4c96-90bd-2ab711dce4b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:04.947193 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c173e483-9d39-4c96-90bd-2ab711dce4b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:04.947618 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c173e483-9d39-4c96-90bd-2ab711dce4b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:04.975481 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c173e483-9d39-4c96-90bd-2ab711dce4b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.039833 Jun 10 13:10:04.976015 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 344/677] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:04 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:05.950443 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5dc4dfdc-054a-4d8a-88f1-c806434f94b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:05.951044 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5dc4dfdc-054a-4d8a-88f1-c806434f94b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:05.957548 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5dc4dfdc-054a-4d8a-88f1-c806434f94b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:05.957985 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5dc4dfdc-054a-4d8a-88f1-c806434f94b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:05.958741 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5dc4dfdc-054a-4d8a-88f1-c806434f94b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:05.959153 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5dc4dfdc-054a-4d8a-88f1-c806434f94b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:06.083514 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5dc4dfdc-054a-4d8a-88f1-c806434f94b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:06.091900 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5dc4dfdc-054a-4d8a-88f1-c806434f94b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.145235 Jun 10 13:10:06.092461 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 345/678] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:05 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:10:06.097006 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c793c663-f664-48fb-be9f-66791515bc58 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:06.097847 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c793c663-f664-48fb-be9f-66791515bc58 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:06.104038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c793c663-f664-48fb-be9f-66791515bc58 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:06.104480 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c793c663-f664-48fb-be9f-66791515bc58 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:06.105260 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c793c663-f664-48fb-be9f-66791515bc58 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:06.105693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c793c663-f664-48fb-be9f-66791515bc58 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:06.133326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c793c663-f664-48fb-be9f-66791515bc58 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.039874 Jun 10 13:10:06.133875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 346/679] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:06 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:07.109642 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3dadfac1-1039-47b2-b883-412a09b28fe1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:07.110321 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3dadfac1-1039-47b2-b883-412a09b28fe1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:07.117107 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3dadfac1-1039-47b2-b883-412a09b28fe1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:07.117554 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3dadfac1-1039-47b2-b883-412a09b28fe1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:07.118311 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3dadfac1-1039-47b2-b883-412a09b28fe1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:07.118739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3dadfac1-1039-47b2-b883-412a09b28fe1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:07.248498 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3dadfac1-1039-47b2-b883-412a09b28fe1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:07.256013 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3dadfac1-1039-47b2-b883-412a09b28fe1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.150214 Jun 10 13:10:07.256581 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 347/680] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:07 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:10:07.261350 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f47a5942-c3c8-463a-a449-2fbc0025bced tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:07.262156 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f47a5942-c3c8-463a-a449-2fbc0025bced tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:07.267939 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f47a5942-c3c8-463a-a449-2fbc0025bced tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:07.268384 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f47a5942-c3c8-463a-a449-2fbc0025bced tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:07.269172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f47a5942-c3c8-463a-a449-2fbc0025bced tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:07.269603 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f47a5942-c3c8-463a-a449-2fbc0025bced tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:07.298034 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f47a5942-c3c8-463a-a449-2fbc0025bced tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040443 Jun 10 13:10:07.298573 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 348/681] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:07 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:08.273162 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4137ca2f-2f22-4531-8743-a6414040c42d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:08.273795 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4137ca2f-2f22-4531-8743-a6414040c42d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:08.280283 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4137ca2f-2f22-4531-8743-a6414040c42d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:08.280725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4137ca2f-2f22-4531-8743-a6414040c42d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:08.281516 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4137ca2f-2f22-4531-8743-a6414040c42d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:08.281941 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4137ca2f-2f22-4531-8743-a6414040c42d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:08.412037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4137ca2f-2f22-4531-8743-a6414040c42d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:08.419590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4137ca2f-2f22-4531-8743-a6414040c42d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.150214 Jun 10 13:10:08.420154 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 349/682] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:08 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:10:08.424609 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-30185c3d-f325-403e-9897-71a788889349 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:08.425526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-30185c3d-f325-403e-9897-71a788889349 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:08.431138 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-30185c3d-f325-403e-9897-71a788889349 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:08.432088 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-30185c3d-f325-403e-9897-71a788889349 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:08.432867 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-30185c3d-f325-403e-9897-71a788889349 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:08.433299 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-30185c3d-f325-403e-9897-71a788889349 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:08.461457 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-30185c3d-f325-403e-9897-71a788889349 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040247 Jun 10 13:10:08.461929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 350/683] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:08 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:09.435813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0266e306-7141-4b63-a182-606561b28d14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:09.436414 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0266e306-7141-4b63-a182-606561b28d14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:09.443652 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0266e306-7141-4b63-a182-606561b28d14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:09.444097 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0266e306-7141-4b63-a182-606561b28d14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:09.444881 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0266e306-7141-4b63-a182-606561b28d14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:09.445314 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0266e306-7141-4b63-a182-606561b28d14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:09.569097 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0266e306-7141-4b63-a182-606561b28d14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:09.577001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0266e306-7141-4b63-a182-606561b28d14 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.145022 Jun 10 13:10:09.577561 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 351/684] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:09 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:10:09.582284 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f556fde1-639c-4a46-bd31-5e28e59d0700 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:09.583182 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f556fde1-639c-4a46-bd31-5e28e59d0700 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:09.589785 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f556fde1-639c-4a46-bd31-5e28e59d0700 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:09.590320 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f556fde1-639c-4a46-bd31-5e28e59d0700 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:09.591103 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f556fde1-639c-4a46-bd31-5e28e59d0700 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:09.591530 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f556fde1-639c-4a46-bd31-5e28e59d0700 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:09.620577 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f556fde1-639c-4a46-bd31-5e28e59d0700 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042055 Jun 10 13:10:09.621151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 352/685] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:09 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:10.595004 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9a09cf12-2d96-4be2-a148-b24c5278b778 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:10.595650 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9a09cf12-2d96-4be2-a148-b24c5278b778 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:10.602023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9a09cf12-2d96-4be2-a148-b24c5278b778 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:10.602470 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9a09cf12-2d96-4be2-a148-b24c5278b778 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:10.603867 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9a09cf12-2d96-4be2-a148-b24c5278b778 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:10.604317 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9a09cf12-2d96-4be2-a148-b24c5278b778 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:10.726699 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9a09cf12-2d96-4be2-a148-b24c5278b778 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:10.734712 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9a09cf12-2d96-4be2-a148-b24c5278b778 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.143605 Jun 10 13:10:10.735290 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 353/686] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:10 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:10:10.739987 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-88414b01-6245-462a-b2e5-10126ba3392a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:10.740909 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-88414b01-6245-462a-b2e5-10126ba3392a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:10.746504 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-88414b01-6245-462a-b2e5-10126ba3392a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:10.746950 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-88414b01-6245-462a-b2e5-10126ba3392a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:10.747710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-88414b01-6245-462a-b2e5-10126ba3392a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:10.748145 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-88414b01-6245-462a-b2e5-10126ba3392a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:10.776600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-88414b01-6245-462a-b2e5-10126ba3392a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040332 Jun 10 13:10:10.777173 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 354/687] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:10 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:11.752151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b3693727-d2dc-42c0-898c-cf9d9e2b2e58 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:11.752786 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b3693727-d2dc-42c0-898c-cf9d9e2b2e58 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:11.759349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b3693727-d2dc-42c0-898c-cf9d9e2b2e58 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:11.759824 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b3693727-d2dc-42c0-898c-cf9d9e2b2e58 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:11.760571 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b3693727-d2dc-42c0-898c-cf9d9e2b2e58 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:11.761027 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b3693727-d2dc-42c0-898c-cf9d9e2b2e58 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:11.887152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b3693727-d2dc-42c0-898c-cf9d9e2b2e58 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:11.894517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b3693727-d2dc-42c0-898c-cf9d9e2b2e58 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.146237 Jun 10 13:10:11.895075 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 355/688] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:11 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:10:11.899460 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c3b4c0f4-bab6-4832-b413-c7d8fa3c3acd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:11.900485 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c3b4c0f4-bab6-4832-b413-c7d8fa3c3acd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:11.906080 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c3b4c0f4-bab6-4832-b413-c7d8fa3c3acd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:11.906520 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c3b4c0f4-bab6-4832-b413-c7d8fa3c3acd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:11.907265 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c3b4c0f4-bab6-4832-b413-c7d8fa3c3acd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:11.907694 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c3b4c0f4-bab6-4832-b413-c7d8fa3c3acd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:11.937029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c3b4c0f4-bab6-4832-b413-c7d8fa3c3acd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040942 Jun 10 13:10:11.937584 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 356/689] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:11 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:12.911918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-bab0c227-21b7-466d-981f-5d83adeb8976 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:12.912591 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-bab0c227-21b7-466d-981f-5d83adeb8976 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:12.919000 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bab0c227-21b7-466d-981f-5d83adeb8976 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:12.919448 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bab0c227-21b7-466d-981f-5d83adeb8976 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:12.920208 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bab0c227-21b7-466d-981f-5d83adeb8976 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:12.920642 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bab0c227-21b7-466d-981f-5d83adeb8976 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:13.060664 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-bab0c227-21b7-466d-981f-5d83adeb8976 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:13.067277 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-bab0c227-21b7-466d-981f-5d83adeb8976 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.159270 Jun 10 13:10:13.067839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 357/690] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:12 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:10:13.175360 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-83833651-a304-4fd8-9c02-65dfa2418ced tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:13.176360 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-83833651-a304-4fd8-9c02-65dfa2418ced tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:13.183069 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-83833651-a304-4fd8-9c02-65dfa2418ced tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:13.183513 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-83833651-a304-4fd8-9c02-65dfa2418ced tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:13.184268 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-83833651-a304-4fd8-9c02-65dfa2418ced tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:13.184701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-83833651-a304-4fd8-9c02-65dfa2418ced tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:13.212291 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-83833651-a304-4fd8-9c02-65dfa2418ced tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.143479 Jun 10 13:10:13.212875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 358/691] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:13 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:14.083550 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e6fbc5ab-a9a1-487b-a302-6b553f2f751f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:14.084149 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e6fbc5ab-a9a1-487b-a302-6b553f2f751f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:14.091164 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6fbc5ab-a9a1-487b-a302-6b553f2f751f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:14.091608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6fbc5ab-a9a1-487b-a302-6b553f2f751f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:14.092356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6fbc5ab-a9a1-487b-a302-6b553f2f751f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:14.092813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6fbc5ab-a9a1-487b-a302-6b553f2f751f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:14.252130 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e6fbc5ab-a9a1-487b-a302-6b553f2f751f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:14.255954 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:14.268164 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e6fbc5ab-a9a1-487b-a302-6b553f2f751f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.181457 Jun 10 13:10:14.268164 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 359/692] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:14 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:10:14.268164 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8169fb9b-190d-47de-8c7e-2eadee7be69d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:14.268164 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8169fb9b-190d-47de-8c7e-2eadee7be69d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:14.274181 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8169fb9b-190d-47de-8c7e-2eadee7be69d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:14.274638 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8169fb9b-190d-47de-8c7e-2eadee7be69d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:14.275619 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8169fb9b-190d-47de-8c7e-2eadee7be69d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:14.276064 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8169fb9b-190d-47de-8c7e-2eadee7be69d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:14.309114 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8169fb9b-190d-47de-8c7e-2eadee7be69d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045687 Jun 10 13:10:14.309693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 360/693] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:14 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:15.091719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ebaedba6-ea7b-4da2-83d7-557ad4360a0a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "POST /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 31.451311 Jun 10 13:10:15.091719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 334/694] 10.222.0.25 () {60 vars in 1322 bytes} [Fri Jun 10 13:09:43 2022] POST /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f/os-volume_attachments => generated 194 bytes in 31453 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:10:15.278576 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ff7fc83d-0bb9-45a7-ab88-c5165d818fca tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:15.279190 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ff7fc83d-0bb9-45a7-ab88-c5165d818fca tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:15.285885 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ff7fc83d-0bb9-45a7-ab88-c5165d818fca tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:15.286337 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ff7fc83d-0bb9-45a7-ab88-c5165d818fca tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:15.287098 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ff7fc83d-0bb9-45a7-ab88-c5165d818fca tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:15.287527 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ff7fc83d-0bb9-45a7-ab88-c5165d818fca tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:15.321855 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2fa36e48-435a-4131-a8e4-c4ab9e1ea92c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:15.324085 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2fa36e48-435a-4131-a8e4-c4ab9e1ea92c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:15.331400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2fa36e48-435a-4131-a8e4-c4ab9e1ea92c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:15.331975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2fa36e48-435a-4131-a8e4-c4ab9e1ea92c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:15.332742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2fa36e48-435a-4131-a8e4-c4ab9e1ea92c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:15.333196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2fa36e48-435a-4131-a8e4-c4ab9e1ea92c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:15.363534 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2fa36e48-435a-4131-a8e4-c4ab9e1ea92c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046362 Jun 10 13:10:15.364093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 335/695] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:15 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:15.498946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ff7fc83d-0bb9-45a7-ab88-c5165d818fca tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:15.506457 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ff7fc83d-0bb9-45a7-ab88-c5165d818fca tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1674 microversion: 2.1 time: 0.232070 Jun 10 13:10:15.507022 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 361/696] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:15 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1674 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:10:15.915152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-a3380d45-2498-44e4-910d-1d0c9eabca31 req-4072c4c2-dce4-4539-82a2-2ac6033740ff service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9e403cf8-5aee-49f3-bbbc-600263bd148a", "name": "network-vif-deleted", "tag": "9615dc48-0838-44bb-9298-f3df821b26c7"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:10:15.929727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-a3380d45-2498-44e4-910d-1d0c9eabca31 req-4072c4c2-dce4-4539-82a2-2ac6033740ff service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:15.947931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-a3380d45-2498-44e4-910d-1d0c9eabca31 req-4072c4c2-dce4-4539-82a2-2ac6033740ff service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:15.947931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-a3380d45-2498-44e4-910d-1d0c9eabca31 req-4072c4c2-dce4-4539-82a2-2ac6033740ff service nova] Creating event network-vif-deleted:9615dc48-0838-44bb-9298-f3df821b26c7 for instance 9e403cf8-5aee-49f3-bbbc-600263bd148a on hv-cinder-maste Jun 10 13:10:15.948881 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-a3380d45-2498-44e4-910d-1d0c9eabca31 req-4072c4c2-dce4-4539-82a2-2ac6033740ff service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038504 Jun 10 13:10:15.949593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 336/697] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:10:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:10:16.376008 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f06f1dfc-4d4d-4ea4-956b-c4d0678a6e29 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:16.377150 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f06f1dfc-4d4d-4ea4-956b-c4d0678a6e29 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:16.384213 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f06f1dfc-4d4d-4ea4-956b-c4d0678a6e29 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:16.384799 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f06f1dfc-4d4d-4ea4-956b-c4d0678a6e29 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:16.385707 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f06f1dfc-4d4d-4ea4-956b-c4d0678a6e29 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:16.386243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f06f1dfc-4d4d-4ea4-956b-c4d0678a6e29 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:16.417704 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f06f1dfc-4d4d-4ea4-956b-c4d0678a6e29 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045859 Jun 10 13:10:16.418436 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 362/698] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:16 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:10:16.524140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9a4cb7fe-d4c8-40ad-95c0-1bed75a18f92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:16.525576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9a4cb7fe-d4c8-40ad-95c0-1bed75a18f92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:16.535023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a4cb7fe-d4c8-40ad-95c0-1bed75a18f92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:16.535023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a4cb7fe-d4c8-40ad-95c0-1bed75a18f92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:16.535226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a4cb7fe-d4c8-40ad-95c0-1bed75a18f92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:16.535667 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a4cb7fe-d4c8-40ad-95c0-1bed75a18f92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:16.575940 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a4cb7fe-d4c8-40ad-95c0-1bed75a18f92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:16.576432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a4cb7fe-d4c8-40ad-95c0-1bed75a18f92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:16.576871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-9a4cb7fe-d4c8-40ad-95c0-1bed75a18f92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lazy-loading 'fault' on Instance uuid 9e403cf8-5aee-49f3-bbbc-600263bd148a {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:10:16.631976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9a4cb7fe-d4c8-40ad-95c0-1bed75a18f92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:16.643042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9a4cb7fe-d4c8-40ad-95c0-1bed75a18f92 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 200 len: 1328 microversion: 2.1 time: 0.122999 Jun 10 13:10:16.643638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 337/699] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:16 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 1328 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:10:17.429577 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-588d9e10-a52b-480d-8962-910209c73ee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:17.430515 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-588d9e10-a52b-480d-8962-910209c73ee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:17.437541 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-588d9e10-a52b-480d-8962-910209c73ee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:17.438295 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-588d9e10-a52b-480d-8962-910209c73ee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:17.438737 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-588d9e10-a52b-480d-8962-910209c73ee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:17.439176 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-588d9e10-a52b-480d-8962-910209c73ee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:17.467819 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-588d9e10-a52b-480d-8962-910209c73ee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042241 Jun 10 13:10:17.468366 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 363/700] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:17 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:17.762133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-20e6992f-c8a2-4221-8299-a238dad2014c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:17.762743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-20e6992f-c8a2-4221-8299-a238dad2014c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:17.768735 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20e6992f-c8a2-4221-8299-a238dad2014c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:17.769274 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20e6992f-c8a2-4221-8299-a238dad2014c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:17.769980 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20e6992f-c8a2-4221-8299-a238dad2014c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:17.770414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20e6992f-c8a2-4221-8299-a238dad2014c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:17.783215 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-20e6992f-c8a2-4221-8299-a238dad2014c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] HTTP exception thrown: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-20e6992f-c8a2-4221-8299-a238dad2014c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Request method failure captured: Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a HTTP/1.1 Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0GLIJPQ8lSWSCxXU9neIcS9fWNdW4XJF-omQVn8xvgfajtrPtR9xT7yJRQ7uDkAeVSjjX8Zr-TojYB67_YvghXuxNAaWo9VFXLD41gtOICuurNBjdxNv9Y58tQmvsCSdX2FomGdBsssQdaFXDYnlp0d7vxIpJLVub7cC52vynSBvwE Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 6dace38cecd44f87becbf6dc213851b3 Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:10:17.784841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:10:17.786467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_6dace38cecd44f87becbf6dc213851b3", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:10:17.786467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:10:17.786467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 6dace38cecd44f87becbf6dc213851b3 Jun 10 13:10:17.786467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:10:17.786467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeTestJSON-1599090335-project Jun 10 13:10:17.786467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:10:17.786467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:10:17.786467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 9c8b2c27b69a4776ab6b7b7820da02d9 Jun 10 13:10:17.786467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeTestJSON-1599090335-project Jun 10 13:10:17.786467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:10:17.786467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:10:17.786467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '9e403cf8-5aee-49f3-bbbc-600263bd148a'} Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:10:17.787393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:10:17.788984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:10:17.790253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-20e6992f-c8a2-4221-8299-a238dad2014c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Returning 404 to user: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:10:17.790253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-20e6992f-c8a2-4221-8299-a238dad2014c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 404 len: 111 microversion: 2.1 time: 0.129225 Jun 10 13:10:17.790253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 338/701] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:17 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 111 bytes in 130 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:10:17.795514 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3797f17e-0d78-4988-a0ab-862a4a237c2d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '.version_select at 0x7f930513a4c0>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:17.882511 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3797f17e-0d78-4988-a0ab-862a4a237c2d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1040680043" status: 202 len: 0 microversion: 2.1 time: 0.090626 Jun 10 13:10:17.883013 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 364/702] 10.222.0.25 () {58 vars in 1220 bytes} [Fri Jun 10 13:10:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1040680043 => generated 0 bytes in 92 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jun 10 13:10:18.484702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-81bbe179-b9d4-4bb5-a4a7-2a87499365af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:18.485708 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-81bbe179-b9d4-4bb5-a4a7-2a87499365af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:18.492370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-81bbe179-b9d4-4bb5-a4a7-2a87499365af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:18.492842 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-81bbe179-b9d4-4bb5-a4a7-2a87499365af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:18.493627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-81bbe179-b9d4-4bb5-a4a7-2a87499365af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:18.494065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-81bbe179-b9d4-4bb5-a4a7-2a87499365af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:18.526396 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-81bbe179-b9d4-4bb5-a4a7-2a87499365af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045593 Jun 10 13:10:18.527179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 339/703] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:18 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:18.880200 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d7e7f161-e560-4581-a3c2-6e26e5691007 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Action: 'create', calling method: .version_select at 0x7f93051330d0>, body: {"keypair": {"name": "tempest-keypair-513401585", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBN/QYmjPpmkkZ+8kGgQyD6MnsdQ8/Biq3GozQnCbQGWfmNrpd07Ujg/sbwMHf0YPR1FM7jicJPglDyNB0IqROeI0w1462K+qVyA0nw7+LRpbiKs5tA5u0a9tihfxOspgCg=="}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:10:18.895444 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-d7e7f161-e560-4581-a3c2-6e26e5691007 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Getting quotas for project 6dace38cecd44f87becbf6dc213851b3. Resources: {'key_pairs'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:10:18.899263 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.quota [None req-d7e7f161-e560-4581-a3c2-6e26e5691007 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Getting quotas for user 9c8b2c27b69a4776ab6b7b7820da02d9 and project 6dace38cecd44f87becbf6dc213851b3. Resources: {'key_pairs'} {{(pid=118093) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:10:18.913013 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=99,nova_api:DELETE=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:18.991871 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d7e7f161-e560-4581-a3c2-6e26e5691007 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.115999 Jun 10 13:10:18.992494 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 365/704] 10.222.0.25 () {60 vars in 1158 bytes} [Fri Jun 10 13:10:18 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:10:19.537981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a165809e-4715-497d-be93-d9c11801cd86 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:19.538943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a165809e-4715-497d-be93-d9c11801cd86 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:19.545994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a165809e-4715-497d-be93-d9c11801cd86 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:19.546434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a165809e-4715-497d-be93-d9c11801cd86 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:19.547292 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a165809e-4715-497d-be93-d9c11801cd86 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:19.547722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a165809e-4715-497d-be93-d9c11801cd86 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:19.577880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a165809e-4715-497d-be93-d9c11801cd86 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043820 Jun 10 13:10:19.578865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 340/705] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:19 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:20.590234 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cf864d6f-1d83-4955-8747-b10a7d2e5aa5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:20.591181 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cf864d6f-1d83-4955-8747-b10a7d2e5aa5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:20.597811 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cf864d6f-1d83-4955-8747-b10a7d2e5aa5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:20.598262 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cf864d6f-1d83-4955-8747-b10a7d2e5aa5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:20.599127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cf864d6f-1d83-4955-8747-b10a7d2e5aa5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:20.599572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cf864d6f-1d83-4955-8747-b10a7d2e5aa5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:20.647147 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cf864d6f-1d83-4955-8747-b10a7d2e5aa5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061327 Jun 10 13:10:20.647717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 366/706] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:20 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:21.062781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1948417838", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a63fb262-f8e3-4ba4-b86b-b0f4cec54e28"}], "security_groups": [{"name": "tempest-securitygroup--1951192744"}], "key_name": "tempest-keypair-513401585", "user_data": "}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:10:21.164372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.network.neutron [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] validate_networks() for [('a63fb262-f8e3-4ba4-b86b-b0f4cec54e28', None, None, None, None, None)] {{(pid=118092) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:10:21.522143 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Flavor limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:10:21.522567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Image limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:10:21.523041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Flavor pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:10:21.523448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Image pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:10:21.523859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:10:21.533318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:21.533760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:21.535163 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:21.535163 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:21.551436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Getting quotas for project 6dace38cecd44f87becbf6dc213851b3. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:10:21.555378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Getting quotas for user 9c8b2c27b69a4776ab6b7b7820da02d9 and project 6dace38cecd44f87becbf6dc213851b3. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:10:21.566268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Going to run 1 instances... {{(pid=118092) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:10:21.568483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118092) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:10:21.658149 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-884e4451-0fe8-4e81-85ad-f2c05a41856d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:21.659199 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-884e4451-0fe8-4e81-85ad-f2c05a41856d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:21.665618 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-884e4451-0fe8-4e81-85ad-f2c05a41856d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:21.666053 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-884e4451-0fe8-4e81-85ad-f2c05a41856d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:21.666806 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-884e4451-0fe8-4e81-85ad-f2c05a41856d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:21.667232 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-884e4451-0fe8-4e81-85ad-f2c05a41856d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:21.672717 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-133c3b07-b1bb-44b7-a5d4-e5674cbe32de tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.614472 Jun 10 13:10:21.673576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 341/707] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:10:21 2022] POST /compute/v2.1/servers => generated 402 bytes in 616 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jun 10 13:10:21.683779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fb5b4123-807a-4227-a3ad-db2ea0b3d7fb tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:21.684394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fb5b4123-807a-4227-a3ad-db2ea0b3d7fb tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:21.698253 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-884e4451-0fe8-4e81-85ad-f2c05a41856d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043959 Jun 10 13:10:21.698831 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 367/708] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:21 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:10:21.743604 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fb5b4123-807a-4227-a3ad-db2ea0b3d7fb tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:21.751840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fb5b4123-807a-4227-a3ad-db2ea0b3d7fb tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1237 microversion: 2.1 time: 0.072274 Jun 10 13:10:21.752411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 342/709] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:21 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1237 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:10:22.710011 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f19ea315-db17-4aba-8d71-632b9ba26ab0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:22.710914 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f19ea315-db17-4aba-8d71-632b9ba26ab0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:22.718280 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f19ea315-db17-4aba-8d71-632b9ba26ab0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:22.718720 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f19ea315-db17-4aba-8d71-632b9ba26ab0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:22.719475 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f19ea315-db17-4aba-8d71-632b9ba26ab0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:22.719902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f19ea315-db17-4aba-8d71-632b9ba26ab0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:22.748674 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f19ea315-db17-4aba-8d71-632b9ba26ab0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042682 Jun 10 13:10:22.749258 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 368/710] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:22 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:22.769182 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-53df9fe1-9ea0-46e1-a705-10bc29833c3b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:22.769788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-53df9fe1-9ea0-46e1-a705-10bc29833c3b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:22.778252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53df9fe1-9ea0-46e1-a705-10bc29833c3b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:22.778781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53df9fe1-9ea0-46e1-a705-10bc29833c3b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:22.779668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53df9fe1-9ea0-46e1-a705-10bc29833c3b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:22.780407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53df9fe1-9ea0-46e1-a705-10bc29833c3b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:22.846374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-53df9fe1-9ea0-46e1-a705-10bc29833c3b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:22.854000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-53df9fe1-9ea0-46e1-a705-10bc29833c3b tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1241 microversion: 2.1 time: 0.088679 Jun 10 13:10:22.854555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 343/711] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:22 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1241 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:10:23.759736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7bd1ba8f-2221-49b7-8f04-83a1520cf55d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:23.760651 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7bd1ba8f-2221-49b7-8f04-83a1520cf55d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:23.767215 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7bd1ba8f-2221-49b7-8f04-83a1520cf55d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:23.767666 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7bd1ba8f-2221-49b7-8f04-83a1520cf55d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:23.768427 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7bd1ba8f-2221-49b7-8f04-83a1520cf55d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:23.768899 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7bd1ba8f-2221-49b7-8f04-83a1520cf55d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:23.797763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7bd1ba8f-2221-49b7-8f04-83a1520cf55d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042069 Jun 10 13:10:23.798315 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 369/712] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:23 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:23.871935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-781e4869-f722-47f4-aef6-74f9061fb14f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:23.872489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-781e4869-f722-47f4-aef6-74f9061fb14f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:23.878914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-781e4869-f722-47f4-aef6-74f9061fb14f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:23.879400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-781e4869-f722-47f4-aef6-74f9061fb14f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:23.880157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-781e4869-f722-47f4-aef6-74f9061fb14f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:23.880584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-781e4869-f722-47f4-aef6-74f9061fb14f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:23.943230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-781e4869-f722-47f4-aef6-74f9061fb14f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:23.950816 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-781e4869-f722-47f4-aef6-74f9061fb14f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1297 microversion: 2.1 time: 0.083496 Jun 10 13:10:23.951388 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 344/713] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:23 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1297 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:10:24.810219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3f2c3e05-feeb-42a5-9b5a-0a21d05565b9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:24.811142 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3f2c3e05-feeb-42a5-9b5a-0a21d05565b9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:24.817961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3f2c3e05-feeb-42a5-9b5a-0a21d05565b9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:24.818403 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3f2c3e05-feeb-42a5-9b5a-0a21d05565b9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:24.819175 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3f2c3e05-feeb-42a5-9b5a-0a21d05565b9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:24.819606 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3f2c3e05-feeb-42a5-9b5a-0a21d05565b9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:24.850251 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3f2c3e05-feeb-42a5-9b5a-0a21d05565b9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044561 Jun 10 13:10:24.850901 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 370/714] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:24 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:24.967544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b2e16b6c-097b-4e39-ba74-95ea422d05ab tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:24.968152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b2e16b6c-097b-4e39-ba74-95ea422d05ab tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:24.975036 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b2e16b6c-097b-4e39-ba74-95ea422d05ab tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:24.975540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b2e16b6c-097b-4e39-ba74-95ea422d05ab tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:24.977099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b2e16b6c-097b-4e39-ba74-95ea422d05ab tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:24.977733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b2e16b6c-097b-4e39-ba74-95ea422d05ab tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:25.105744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b2e16b6c-097b-4e39-ba74-95ea422d05ab tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:25.113588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b2e16b6c-097b-4e39-ba74-95ea422d05ab tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1363 microversion: 2.1 time: 0.149942 Jun 10 13:10:25.114154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 345/715] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:24 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1363 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:10:25.684927 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-7272ac1d-6795-4ae0-9aec-68fbeb7058d5 req-7958bada-4314-46da-9c27-e9bd0243a2ef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6273d596-6d75-49f6-9c03-17ea7b4422d7", "tag": "d939095b-bad6-443e-ac1d-07e8dd47e777"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:10:25.693213 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-7272ac1d-6795-4ae0-9aec-68fbeb7058d5 req-7958bada-4314-46da-9c27-e9bd0243a2ef service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:25.700062 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-7272ac1d-6795-4ae0-9aec-68fbeb7058d5 req-7958bada-4314-46da-9c27-e9bd0243a2ef service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:25.706647 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-7272ac1d-6795-4ae0-9aec-68fbeb7058d5 req-7958bada-4314-46da-9c27-e9bd0243a2ef service nova] Creating event network-changed:d939095b-bad6-443e-ac1d-07e8dd47e777 for instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 on hv-cinder-maste Jun 10 13:10:25.711845 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-7272ac1d-6795-4ae0-9aec-68fbeb7058d5 req-7958bada-4314-46da-9c27-e9bd0243a2ef service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.098657 Jun 10 13:10:25.712442 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 371/716] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:10:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:10:25.862269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f47ecf73-345a-41b5-98b5-4e392976e503 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:25.863425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f47ecf73-345a-41b5-98b5-4e392976e503 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:25.869882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f47ecf73-345a-41b5-98b5-4e392976e503 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:25.870906 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f47ecf73-345a-41b5-98b5-4e392976e503 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:25.871678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f47ecf73-345a-41b5-98b5-4e392976e503 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:25.872112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f47ecf73-345a-41b5-98b5-4e392976e503 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:25.900952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f47ecf73-345a-41b5-98b5-4e392976e503 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042716 Jun 10 13:10:25.901522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 346/717] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:25 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:26.132294 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8bedefa9-c55a-49f1-9f99-493f6c4e8af1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:26.133120 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8bedefa9-c55a-49f1-9f99-493f6c4e8af1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:26.139398 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8bedefa9-c55a-49f1-9f99-493f6c4e8af1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:26.139917 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8bedefa9-c55a-49f1-9f99-493f6c4e8af1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:26.140681 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8bedefa9-c55a-49f1-9f99-493f6c4e8af1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:26.141208 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8bedefa9-c55a-49f1-9f99-493f6c4e8af1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:26.274491 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8bedefa9-c55a-49f1-9f99-493f6c4e8af1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:26.282776 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8bedefa9-c55a-49f1-9f99-493f6c4e8af1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1363 microversion: 2.1 time: 0.155114 Jun 10 13:10:26.283357 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 372/718] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:26 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1363 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:10:26.911846 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8a4f2629-6cb9-4ea8-afb4-debe639dc091 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:26.912791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8a4f2629-6cb9-4ea8-afb4-debe639dc091 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:26.919207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a4f2629-6cb9-4ea8-afb4-debe639dc091 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:26.919651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a4f2629-6cb9-4ea8-afb4-debe639dc091 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:26.920403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a4f2629-6cb9-4ea8-afb4-debe639dc091 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:26.920861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a4f2629-6cb9-4ea8-afb4-debe639dc091 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:26.950395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8a4f2629-6cb9-4ea8-afb4-debe639dc091 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042519 Jun 10 13:10:26.950951 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 347/719] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:26 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:27.299438 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-02ed9d18-c1bd-4604-9e56-5cb9956ef831 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:27.300053 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-02ed9d18-c1bd-4604-9e56-5cb9956ef831 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:27.306864 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-02ed9d18-c1bd-4604-9e56-5cb9956ef831 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:27.307918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-02ed9d18-c1bd-4604-9e56-5cb9956ef831 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:27.308698 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-02ed9d18-c1bd-4604-9e56-5cb9956ef831 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:27.309164 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-02ed9d18-c1bd-4604-9e56-5cb9956ef831 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:27.537829 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-02ed9d18-c1bd-4604-9e56-5cb9956ef831 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:27.545942 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-02ed9d18-c1bd-4604-9e56-5cb9956ef831 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1363 microversion: 2.1 time: 0.250413 Jun 10 13:10:27.546527 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 373/720] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:27 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1363 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:10:27.962022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bf512a58-9f2a-4a2c-ba80-4a2f7c2c99c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:27.962932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bf512a58-9f2a-4a2c-ba80-4a2f7c2c99c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:27.970018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bf512a58-9f2a-4a2c-ba80-4a2f7c2c99c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:27.970456 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bf512a58-9f2a-4a2c-ba80-4a2f7c2c99c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:27.971211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bf512a58-9f2a-4a2c-ba80-4a2f7c2c99c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:27.971641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bf512a58-9f2a-4a2c-ba80-4a2f7c2c99c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:28.001490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bf512a58-9f2a-4a2c-ba80-4a2f7c2c99c7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043444 Jun 10 13:10:28.002040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 348/721] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:27 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:10:28.563047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8c61d191-5267-4df6-a739-a2e5d77a50d1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:28.563655 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8c61d191-5267-4df6-a739-a2e5d77a50d1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:28.570134 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8c61d191-5267-4df6-a739-a2e5d77a50d1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:28.570582 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8c61d191-5267-4df6-a739-a2e5d77a50d1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:28.571339 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8c61d191-5267-4df6-a739-a2e5d77a50d1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:28.571771 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8c61d191-5267-4df6-a739-a2e5d77a50d1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:28.906218 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell0:SELECT=4 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:28.938634 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8c61d191-5267-4df6-a739-a2e5d77a50d1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:28.946052 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8c61d191-5267-4df6-a739-a2e5d77a50d1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1363 microversion: 2.1 time: 0.386855 Jun 10 13:10:28.946611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 374/722] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:28 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1363 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:10:29.013210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a0e60b33-74a7-4e65-aeb6-8e6bbd9c04aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:29.014122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a0e60b33-74a7-4e65-aeb6-8e6bbd9c04aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:29.020573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0e60b33-74a7-4e65-aeb6-8e6bbd9c04aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:29.021084 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0e60b33-74a7-4e65-aeb6-8e6bbd9c04aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:29.021889 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0e60b33-74a7-4e65-aeb6-8e6bbd9c04aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:29.022321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0e60b33-74a7-4e65-aeb6-8e6bbd9c04aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:29.051250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a0e60b33-74a7-4e65-aeb6-8e6bbd9c04aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042169 Jun 10 13:10:29.052011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 349/723] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:29 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:29.962884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e708810b-52a9-4e95-b182-bf1896355387 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:29.963527 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e708810b-52a9-4e95-b182-bf1896355387 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:29.970907 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e708810b-52a9-4e95-b182-bf1896355387 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:29.971349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e708810b-52a9-4e95-b182-bf1896355387 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:29.972102 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e708810b-52a9-4e95-b182-bf1896355387 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:29.972534 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e708810b-52a9-4e95-b182-bf1896355387 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:30.062952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-209337cb-10c5-4277-9a2e-a7498ba054d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:30.067268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-209337cb-10c5-4277-9a2e-a7498ba054d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:30.073790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-209337cb-10c5-4277-9a2e-a7498ba054d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:30.074232 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-209337cb-10c5-4277-9a2e-a7498ba054d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:30.074991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-209337cb-10c5-4277-9a2e-a7498ba054d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:30.075417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-209337cb-10c5-4277-9a2e-a7498ba054d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:30.099668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e708810b-52a9-4e95-b182-bf1896355387 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:30.105762 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-209337cb-10c5-4277-9a2e-a7498ba054d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046090 Jun 10 13:10:30.106331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 350/724] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:30 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:10:30.107911 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e708810b-52a9-4e95-b182-bf1896355387 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1363 microversion: 2.1 time: 0.148936 Jun 10 13:10:30.108604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 375/725] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:29 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1363 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:10:31.117614 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-689f9037-9d71-456b-b99a-41591d11ffc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:31.118553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-689f9037-9d71-456b-b99a-41591d11ffc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:31.125503 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-20dfccad-7474-4aa8-bfc5-a2b02c5449b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:31.125976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-689f9037-9d71-456b-b99a-41591d11ffc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:31.126079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-689f9037-9d71-456b-b99a-41591d11ffc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:31.126246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-20dfccad-7474-4aa8-bfc5-a2b02c5449b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:31.126832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-689f9037-9d71-456b-b99a-41591d11ffc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:31.127264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-689f9037-9d71-456b-b99a-41591d11ffc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:31.132987 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-20dfccad-7474-4aa8-bfc5-a2b02c5449b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:31.133442 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-20dfccad-7474-4aa8-bfc5-a2b02c5449b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:31.134415 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-20dfccad-7474-4aa8-bfc5-a2b02c5449b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:31.134945 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-20dfccad-7474-4aa8-bfc5-a2b02c5449b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:31.157691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-689f9037-9d71-456b-b99a-41591d11ffc2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044005 Jun 10 13:10:31.158253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 351/726] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:31 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:31.261152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-20dfccad-7474-4aa8-bfc5-a2b02c5449b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:31.268507 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-20dfccad-7474-4aa8-bfc5-a2b02c5449b6 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1363 microversion: 2.1 time: 0.146900 Jun 10 13:10:31.269104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 376/727] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:31 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1363 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:10:31.868207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-9dbb0ea0-16e5-4106-83b6-43717ed75e89 req-fab5af08-7e66-4d6f-b534-7bf556ec0c1e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6273d596-6d75-49f6-9c03-17ea7b4422d7", "name": "network-vif-plugged", "status": "completed", "tag": "d939095b-bad6-443e-ac1d-07e8dd47e777"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:10:31.872410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:31.873259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=8 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:31.878352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-9dbb0ea0-16e5-4106-83b6-43717ed75e89 req-fab5af08-7e66-4d6f-b534-7bf556ec0c1e service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:31.878787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-9dbb0ea0-16e5-4106-83b6-43717ed75e89 req-fab5af08-7e66-4d6f-b534-7bf556ec0c1e service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:31.889407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-9dbb0ea0-16e5-4106-83b6-43717ed75e89 req-fab5af08-7e66-4d6f-b534-7bf556ec0c1e service nova] Creating event network-vif-plugged:d939095b-bad6-443e-ac1d-07e8dd47e777 for instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 on hv-cinder-maste Jun 10 13:10:31.894566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-9dbb0ea0-16e5-4106-83b6-43717ed75e89 req-fab5af08-7e66-4d6f-b534-7bf556ec0c1e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030046 Jun 10 13:10:31.895144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 352/728] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:10:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:10:32.170140 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-027655fd-da4d-4c9b-998e-029f109addc8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:32.171024 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-027655fd-da4d-4c9b-998e-029f109addc8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:32.177592 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-027655fd-da4d-4c9b-998e-029f109addc8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:32.178037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-027655fd-da4d-4c9b-998e-029f109addc8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:32.178791 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-027655fd-da4d-4c9b-998e-029f109addc8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:32.179223 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-027655fd-da4d-4c9b-998e-029f109addc8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:32.209167 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-027655fd-da4d-4c9b-998e-029f109addc8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043434 Jun 10 13:10:32.209733 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 377/729] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:32 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:32.288453 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-135252ea-16c2-4f9f-826f-a78deebf8c4a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:32.289177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-135252ea-16c2-4f9f-826f-a78deebf8c4a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:32.296514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-135252ea-16c2-4f9f-826f-a78deebf8c4a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:32.296995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-135252ea-16c2-4f9f-826f-a78deebf8c4a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:32.297757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-135252ea-16c2-4f9f-826f-a78deebf8c4a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:32.298188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-135252ea-16c2-4f9f-826f-a78deebf8c4a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:32.315944 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:32.429515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-135252ea-16c2-4f9f-826f-a78deebf8c4a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:32.437393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-135252ea-16c2-4f9f-826f-a78deebf8c4a tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1363 microversion: 2.1 time: 0.156013 Jun 10 13:10:32.437968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 353/730] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:32 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1363 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:10:33.221254 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5e197052-feae-40c2-815e-8c9066dd02d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:33.222156 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5e197052-feae-40c2-815e-8c9066dd02d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:33.228602 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5e197052-feae-40c2-815e-8c9066dd02d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:33.229075 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5e197052-feae-40c2-815e-8c9066dd02d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:33.230506 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5e197052-feae-40c2-815e-8c9066dd02d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:33.230945 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5e197052-feae-40c2-815e-8c9066dd02d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:33.259210 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5e197052-feae-40c2-815e-8c9066dd02d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041807 Jun 10 13:10:33.259698 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 378/731] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:33 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:33.454522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0d19674f-c09c-473a-856f-fb293c7fa444 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:33.455205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0d19674f-c09c-473a-856f-fb293c7fa444 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:33.462661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0d19674f-c09c-473a-856f-fb293c7fa444 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:33.463110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0d19674f-c09c-473a-856f-fb293c7fa444 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:33.463977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0d19674f-c09c-473a-856f-fb293c7fa444 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:33.465108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0d19674f-c09c-473a-856f-fb293c7fa444 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:33.593201 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0d19674f-c09c-473a-856f-fb293c7fa444 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:33.599785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0d19674f-c09c-473a-856f-fb293c7fa444 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1363 microversion: 2.1 time: 0.149185 Jun 10 13:10:33.600338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 354/732] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:33 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1363 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:10:34.270617 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9d24ca28-da64-4e37-aec5-a0f59db3bb4c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:34.271534 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9d24ca28-da64-4e37-aec5-a0f59db3bb4c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:34.278570 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9d24ca28-da64-4e37-aec5-a0f59db3bb4c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:34.279016 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9d24ca28-da64-4e37-aec5-a0f59db3bb4c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:34.279770 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9d24ca28-da64-4e37-aec5-a0f59db3bb4c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:34.280202 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9d24ca28-da64-4e37-aec5-a0f59db3bb4c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:34.283821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:34.310054 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9d24ca28-da64-4e37-aec5-a0f59db3bb4c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043374 Jun 10 13:10:34.310606 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 379/733] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:34 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:34.616957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-df92fcf3-3427-41f9-83d8-a4a48f1a8959 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:34.617569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-df92fcf3-3427-41f9-83d8-a4a48f1a8959 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:34.624827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df92fcf3-3427-41f9-83d8-a4a48f1a8959 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:34.625275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df92fcf3-3427-41f9-83d8-a4a48f1a8959 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:34.626044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df92fcf3-3427-41f9-83d8-a4a48f1a8959 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:34.626499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df92fcf3-3427-41f9-83d8-a4a48f1a8959 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:34.753998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-df92fcf3-3427-41f9-83d8-a4a48f1a8959 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:10:34.761873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-df92fcf3-3427-41f9-83d8-a4a48f1a8959 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1550 microversion: 2.1 time: 0.148510 Jun 10 13:10:34.762427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 355/734] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:10:34 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1550 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:10:34.778605 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e9bfdecc-15f4-4cf5-89ca-347be6e41b66 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:34.779084 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e9bfdecc-15f4-4cf5-89ca-347be6e41b66 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:34.785413 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9bfdecc-15f4-4cf5-89ca-347be6e41b66 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:34.785854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9bfdecc-15f4-4cf5-89ca-347be6e41b66 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:34.786602 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9bfdecc-15f4-4cf5-89ca-347be6e41b66 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:34.787029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9bfdecc-15f4-4cf5-89ca-347be6e41b66 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:34.862139 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e9bfdecc-15f4-4cf5-89ca-347be6e41b66 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.088057 Jun 10 13:10:34.862984 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 380/735] 10.222.0.25 () {58 vars in 1274 bytes} [Fri Jun 10 13:10:34 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7/os-interface => generated 285 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:10:35.321861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-15c95ee5-8e39-4320-a6db-1ae42f32779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:35.322902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-15c95ee5-8e39-4320-a6db-1ae42f32779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:35.329836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15c95ee5-8e39-4320-a6db-1ae42f32779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:35.330302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15c95ee5-8e39-4320-a6db-1ae42f32779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:35.331082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15c95ee5-8e39-4320-a6db-1ae42f32779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:35.331553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15c95ee5-8e39-4320-a6db-1ae42f32779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:35.359975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-15c95ee5-8e39-4320-a6db-1ae42f32779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042165 Jun 10 13:10:35.360530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 356/736] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:35 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:35.805453 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-69aa36ce-b9d3-4789-80a0-e46f3ca3ce2e req-d303fe0b-3cfa-4610-bcbf-dddf1d61fee2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6273d596-6d75-49f6-9c03-17ea7b4422d7", "tag": "d939095b-bad6-443e-ac1d-07e8dd47e777"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:10:35.814405 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-69aa36ce-b9d3-4789-80a0-e46f3ca3ce2e req-d303fe0b-3cfa-4610-bcbf-dddf1d61fee2 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:35.814845 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-69aa36ce-b9d3-4789-80a0-e46f3ca3ce2e req-d303fe0b-3cfa-4610-bcbf-dddf1d61fee2 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:35.826156 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-69aa36ce-b9d3-4789-80a0-e46f3ca3ce2e req-d303fe0b-3cfa-4610-bcbf-dddf1d61fee2 service nova] Creating event network-changed:d939095b-bad6-443e-ac1d-07e8dd47e777 for instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 on hv-cinder-maste Jun 10 13:10:35.831272 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-69aa36ce-b9d3-4789-80a0-e46f3ca3ce2e req-d303fe0b-3cfa-4610-bcbf-dddf1d61fee2 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030060 Jun 10 13:10:35.832029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 381/737] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:10:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:10:36.372672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6e462f06-4da0-4a94-b80a-6a4406a7e5f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:36.373729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6e462f06-4da0-4a94-b80a-6a4406a7e5f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:36.380528 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e462f06-4da0-4a94-b80a-6a4406a7e5f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:36.381085 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e462f06-4da0-4a94-b80a-6a4406a7e5f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:36.382072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e462f06-4da0-4a94-b80a-6a4406a7e5f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:36.382587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e462f06-4da0-4a94-b80a-6a4406a7e5f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:36.411919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6e462f06-4da0-4a94-b80a-6a4406a7e5f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044100 Jun 10 13:10:36.412488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 357/738] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:36 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:37.424012 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-81281773-950a-465b-8267-c0ccdb82da1d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:37.425023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-81281773-950a-465b-8267-c0ccdb82da1d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:37.431665 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-81281773-950a-465b-8267-c0ccdb82da1d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:37.432106 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-81281773-950a-465b-8267-c0ccdb82da1d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:37.433377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-81281773-950a-465b-8267-c0ccdb82da1d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:37.433809 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-81281773-950a-465b-8267-c0ccdb82da1d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:37.462465 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-81281773-950a-465b-8267-c0ccdb82da1d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042678 Jun 10 13:10:37.462939 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 382/739] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:37 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:38.472963 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-767a9e80-fa00-4898-bae0-a21b65e9e2a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:38.473926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-767a9e80-fa00-4898-bae0-a21b65e9e2a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:38.480405 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-767a9e80-fa00-4898-bae0-a21b65e9e2a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:38.480893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-767a9e80-fa00-4898-bae0-a21b65e9e2a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:38.481661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-767a9e80-fa00-4898-bae0-a21b65e9e2a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:38.482092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-767a9e80-fa00-4898-bae0-a21b65e9e2a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:38.510927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-767a9e80-fa00-4898-bae0-a21b65e9e2a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041899 Jun 10 13:10:38.512139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 358/740] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:38 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:39.522485 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-77890d83-21dc-42e9-be1f-3e089c23a239 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:39.523395 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-77890d83-21dc-42e9-be1f-3e089c23a239 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:39.529949 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-77890d83-21dc-42e9-be1f-3e089c23a239 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:39.530398 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-77890d83-21dc-42e9-be1f-3e089c23a239 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:39.531151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-77890d83-21dc-42e9-be1f-3e089c23a239 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:39.531590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-77890d83-21dc-42e9-be1f-3e089c23a239 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:39.558999 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-77890d83-21dc-42e9-be1f-3e089c23a239 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040655 Jun 10 13:10:39.559540 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 383/741] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:39 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:40.571080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7cf8318f-ca69-4452-870a-23c635274a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:40.572004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7cf8318f-ca69-4452-870a-23c635274a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:40.578583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7cf8318f-ca69-4452-870a-23c635274a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:40.579029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7cf8318f-ca69-4452-870a-23c635274a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:40.579784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7cf8318f-ca69-4452-870a-23c635274a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:40.580217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7cf8318f-ca69-4452-870a-23c635274a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:40.612251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7cf8318f-ca69-4452-870a-23c635274a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045356 Jun 10 13:10:40.613000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 359/742] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:40 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:41.624837 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e357b96a-ab92-46f4-a1bb-ec68ceadd97d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:41.625678 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e357b96a-ab92-46f4-a1bb-ec68ceadd97d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:41.632205 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e357b96a-ab92-46f4-a1bb-ec68ceadd97d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:41.632651 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e357b96a-ab92-46f4-a1bb-ec68ceadd97d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:41.633445 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e357b96a-ab92-46f4-a1bb-ec68ceadd97d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:41.633881 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e357b96a-ab92-46f4-a1bb-ec68ceadd97d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:41.661447 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e357b96a-ab92-46f4-a1bb-ec68ceadd97d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041504 Jun 10 13:10:41.662011 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 384/743] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:41 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:42.672650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-edadca37-bb03-4c24-8169-80b3c8795a2f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:42.673517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-edadca37-bb03-4c24-8169-80b3c8795a2f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:42.679973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-edadca37-bb03-4c24-8169-80b3c8795a2f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:42.680411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-edadca37-bb03-4c24-8169-80b3c8795a2f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:42.681192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-edadca37-bb03-4c24-8169-80b3c8795a2f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:42.681621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-edadca37-bb03-4c24-8169-80b3c8795a2f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:42.711616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-edadca37-bb03-4c24-8169-80b3c8795a2f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042888 Jun 10 13:10:42.712149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 360/744] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:42 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:43.723003 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d206a025-0b62-412f-925b-4c0f84555380 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:43.723917 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d206a025-0b62-412f-925b-4c0f84555380 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:43.730408 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d206a025-0b62-412f-925b-4c0f84555380 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:43.731350 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d206a025-0b62-412f-925b-4c0f84555380 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:43.732110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d206a025-0b62-412f-925b-4c0f84555380 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:43.732547 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d206a025-0b62-412f-925b-4c0f84555380 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:43.760996 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d206a025-0b62-412f-925b-4c0f84555380 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041864 Jun 10 13:10:43.761544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 385/745] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:43 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:44.775807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-449ffc15-4a5c-4067-8938-2ce4763c3323 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:44.776851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-449ffc15-4a5c-4067-8938-2ce4763c3323 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:44.779844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=70,nova_api:INSERT=3 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:44.784911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-449ffc15-4a5c-4067-8938-2ce4763c3323 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:44.789891 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-449ffc15-4a5c-4067-8938-2ce4763c3323 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:44.789891 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-449ffc15-4a5c-4067-8938-2ce4763c3323 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:44.789891 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-449ffc15-4a5c-4067-8938-2ce4763c3323 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:44.815437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-449ffc15-4a5c-4067-8938-2ce4763c3323 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043862 Jun 10 13:10:44.815986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 361/746] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:44 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:45.827251 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-25729763-3ce4-4289-9eb7-ac1e70ea88f9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:45.828160 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-25729763-3ce4-4289-9eb7-ac1e70ea88f9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:45.834687 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-25729763-3ce4-4289-9eb7-ac1e70ea88f9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:45.835131 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-25729763-3ce4-4289-9eb7-ac1e70ea88f9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:45.835883 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-25729763-3ce4-4289-9eb7-ac1e70ea88f9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:45.836309 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-25729763-3ce4-4289-9eb7-ac1e70ea88f9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:45.864191 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-25729763-3ce4-4289-9eb7-ac1e70ea88f9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041468 Jun 10 13:10:45.864733 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 386/747] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:45 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:46.875769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-96942c20-a391-472e-826f-a75d46b91d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:46.876606 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-96942c20-a391-472e-826f-a75d46b91d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:46.883531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96942c20-a391-472e-826f-a75d46b91d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:46.883969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96942c20-a391-472e-826f-a75d46b91d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:46.884718 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96942c20-a391-472e-826f-a75d46b91d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:46.885177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96942c20-a391-472e-826f-a75d46b91d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:46.915859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-96942c20-a391-472e-826f-a75d46b91d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044141 Jun 10 13:10:46.916404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 362/748] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:46 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:47.927842 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-fbc79f81-0f25-45cc-a4f7-ada3efd797e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:47.928756 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-fbc79f81-0f25-45cc-a4f7-ada3efd797e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:47.935247 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fbc79f81-0f25-45cc-a4f7-ada3efd797e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:47.935692 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fbc79f81-0f25-45cc-a4f7-ada3efd797e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:47.936446 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fbc79f81-0f25-45cc-a4f7-ada3efd797e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:47.936918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fbc79f81-0f25-45cc-a4f7-ada3efd797e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:47.965104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-fbc79f81-0f25-45cc-a4f7-ada3efd797e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041137 Jun 10 13:10:47.965550 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 387/749] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:47 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:48.977255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-477e4177-1447-4ed1-8531-b82ca71a641b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:48.979743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-477e4177-1447-4ed1-8531-b82ca71a641b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:48.985502 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-477e4177-1447-4ed1-8531-b82ca71a641b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:48.985951 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-477e4177-1447-4ed1-8531-b82ca71a641b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:48.986826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-477e4177-1447-4ed1-8531-b82ca71a641b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:48.987260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-477e4177-1447-4ed1-8531-b82ca71a641b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:49.016921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-477e4177-1447-4ed1-8531-b82ca71a641b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043631 Jun 10 13:10:49.017474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 363/750] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:48 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:50.027647 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4d3eba00-1653-4715-aaa8-024f691d4a3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:50.028570 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4d3eba00-1653-4715-aaa8-024f691d4a3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:50.037034 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4d3eba00-1653-4715-aaa8-024f691d4a3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:50.037034 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4d3eba00-1653-4715-aaa8-024f691d4a3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:50.037034 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4d3eba00-1653-4715-aaa8-024f691d4a3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:50.037411 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4d3eba00-1653-4715-aaa8-024f691d4a3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:50.065814 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4d3eba00-1653-4715-aaa8-024f691d4a3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042163 Jun 10 13:10:50.066356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 388/751] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:50 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:51.077488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9d6daa71-06b0-4d3a-9e23-eabb26455550 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:51.078398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9d6daa71-06b0-4d3a-9e23-eabb26455550 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:51.084957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9d6daa71-06b0-4d3a-9e23-eabb26455550 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:51.085407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9d6daa71-06b0-4d3a-9e23-eabb26455550 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:51.086164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9d6daa71-06b0-4d3a-9e23-eabb26455550 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:51.086598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9d6daa71-06b0-4d3a-9e23-eabb26455550 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:51.115942 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9d6daa71-06b0-4d3a-9e23-eabb26455550 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042556 Jun 10 13:10:51.116488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 364/752] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:51 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:52.127972 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a15b76db-a889-4b11-9dee-d0a82ca6ea3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:52.128944 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a15b76db-a889-4b11-9dee-d0a82ca6ea3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:52.131433 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:INSERT=1,nova_api:SELECT=47 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:52.136117 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a15b76db-a889-4b11-9dee-d0a82ca6ea3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:52.136565 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a15b76db-a889-4b11-9dee-d0a82ca6ea3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:52.137373 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a15b76db-a889-4b11-9dee-d0a82ca6ea3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:52.137810 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a15b76db-a889-4b11-9dee-d0a82ca6ea3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:52.166730 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a15b76db-a889-4b11-9dee-d0a82ca6ea3a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043254 Jun 10 13:10:52.167279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 389/753] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:52 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:53.178427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7121e6b7-8d9b-44b9-b3a9-d7fdfe3c6f2f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:53.179318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7121e6b7-8d9b-44b9-b3a9-d7fdfe3c6f2f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:53.185856 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7121e6b7-8d9b-44b9-b3a9-d7fdfe3c6f2f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:53.186323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7121e6b7-8d9b-44b9-b3a9-d7fdfe3c6f2f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:53.187070 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7121e6b7-8d9b-44b9-b3a9-d7fdfe3c6f2f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:53.187499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7121e6b7-8d9b-44b9-b3a9-d7fdfe3c6f2f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:53.215197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7121e6b7-8d9b-44b9-b3a9-d7fdfe3c6f2f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040767 Jun 10 13:10:53.215733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 365/754] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:53 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:54.226215 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c9e808ff-28ad-48bc-90b9-4e2e755f2225 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:54.227125 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c9e808ff-28ad-48bc-90b9-4e2e755f2225 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:54.234396 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c9e808ff-28ad-48bc-90b9-4e2e755f2225 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:54.234836 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c9e808ff-28ad-48bc-90b9-4e2e755f2225 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:54.235588 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c9e808ff-28ad-48bc-90b9-4e2e755f2225 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:54.236018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c9e808ff-28ad-48bc-90b9-4e2e755f2225 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:54.264376 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c9e808ff-28ad-48bc-90b9-4e2e755f2225 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042048 Jun 10 13:10:54.264942 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 390/755] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:54 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:55.275902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b002d7c6-c7fd-4a23-91bd-94f21a042424 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:55.276863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b002d7c6-c7fd-4a23-91bd-94f21a042424 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:55.283833 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b002d7c6-c7fd-4a23-91bd-94f21a042424 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:55.284283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b002d7c6-c7fd-4a23-91bd-94f21a042424 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:55.285070 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b002d7c6-c7fd-4a23-91bd-94f21a042424 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:55.285507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b002d7c6-c7fd-4a23-91bd-94f21a042424 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:55.313764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b002d7c6-c7fd-4a23-91bd-94f21a042424 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041837 Jun 10 13:10:55.314308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 366/756] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:55 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:56.325443 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1b20d0bc-268b-48e6-8685-98e91e5d4dbf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:56.326348 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1b20d0bc-268b-48e6-8685-98e91e5d4dbf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:56.333157 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1b20d0bc-268b-48e6-8685-98e91e5d4dbf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:56.333606 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1b20d0bc-268b-48e6-8685-98e91e5d4dbf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:56.334376 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1b20d0bc-268b-48e6-8685-98e91e5d4dbf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:56.334811 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1b20d0bc-268b-48e6-8685-98e91e5d4dbf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:56.365125 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1b20d0bc-268b-48e6-8685-98e91e5d4dbf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043582 Jun 10 13:10:56.365676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 391/757] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:56 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:57.376959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cdb0ff08-3176-49b0-9002-e75d6afd4b88 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:57.377820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cdb0ff08-3176-49b0-9002-e75d6afd4b88 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:57.384289 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cdb0ff08-3176-49b0-9002-e75d6afd4b88 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:57.384743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cdb0ff08-3176-49b0-9002-e75d6afd4b88 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:57.385569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cdb0ff08-3176-49b0-9002-e75d6afd4b88 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:57.386003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cdb0ff08-3176-49b0-9002-e75d6afd4b88 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:57.414217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cdb0ff08-3176-49b0-9002-e75d6afd4b88 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041953 Jun 10 13:10:57.414764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 367/758] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:57 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:58.426085 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-060921f7-11fc-44a4-9234-9d8b7312973a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:58.426992 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-060921f7-11fc-44a4-9234-9d8b7312973a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:58.433346 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-060921f7-11fc-44a4-9234-9d8b7312973a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:58.433819 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-060921f7-11fc-44a4-9234-9d8b7312973a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:58.434564 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-060921f7-11fc-44a4-9234-9d8b7312973a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:58.434991 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-060921f7-11fc-44a4-9234-9d8b7312973a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:58.464560 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-060921f7-11fc-44a4-9234-9d8b7312973a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042325 Jun 10 13:10:58.465155 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 392/759] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:58 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:10:59.476171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0fd541f5-53b0-4ae1-82cd-04e7df095403 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:10:59.477115 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0fd541f5-53b0-4ae1-82cd-04e7df095403 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:10:59.484102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fd541f5-53b0-4ae1-82cd-04e7df095403 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:59.484545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fd541f5-53b0-4ae1-82cd-04e7df095403 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:59.485335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fd541f5-53b0-4ae1-82cd-04e7df095403 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:10:59.485767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fd541f5-53b0-4ae1-82cd-04e7df095403 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:10:59.514195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0fd541f5-53b0-4ae1-82cd-04e7df095403 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041910 Jun 10 13:10:59.514727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 368/760] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:10:59 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:00.526542 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9282fcc0-eeba-4d7e-b3ed-12065f858d4f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:00.527446 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9282fcc0-eeba-4d7e-b3ed-12065f858d4f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:00.534585 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9282fcc0-eeba-4d7e-b3ed-12065f858d4f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:00.535026 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9282fcc0-eeba-4d7e-b3ed-12065f858d4f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:00.535775 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9282fcc0-eeba-4d7e-b3ed-12065f858d4f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:00.536203 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9282fcc0-eeba-4d7e-b3ed-12065f858d4f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:00.564911 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9282fcc0-eeba-4d7e-b3ed-12065f858d4f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042334 Jun 10 13:11:00.565464 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 393/761] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:00 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:11:01.579220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d18d9b75-7a16-44f1-9e79-3783abcc9d5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:01.580138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d18d9b75-7a16-44f1-9e79-3783abcc9d5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:01.586353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d18d9b75-7a16-44f1-9e79-3783abcc9d5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:01.586748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d18d9b75-7a16-44f1-9e79-3783abcc9d5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:01.587507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d18d9b75-7a16-44f1-9e79-3783abcc9d5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:01.587942 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d18d9b75-7a16-44f1-9e79-3783abcc9d5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:01.615874 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d18d9b75-7a16-44f1-9e79-3783abcc9d5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041138 Jun 10 13:11:01.616417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 369/762] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:01 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:02.627286 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-77e42f90-7da2-4f50-a877-b7e9a00d4079 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:02.628206 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-77e42f90-7da2-4f50-a877-b7e9a00d4079 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:02.634694 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-77e42f90-7da2-4f50-a877-b7e9a00d4079 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:02.635140 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-77e42f90-7da2-4f50-a877-b7e9a00d4079 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:02.635900 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-77e42f90-7da2-4f50-a877-b7e9a00d4079 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:02.636377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-77e42f90-7da2-4f50-a877-b7e9a00d4079 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:02.665514 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-77e42f90-7da2-4f50-a877-b7e9a00d4079 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042216 Jun 10 13:11:02.666054 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 394/763] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:02 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:03.676824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8b207942-b8a4-4106-925b-6d6afcc474cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:03.677728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8b207942-b8a4-4106-925b-6d6afcc474cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:03.684193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8b207942-b8a4-4106-925b-6d6afcc474cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:03.684636 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8b207942-b8a4-4106-925b-6d6afcc474cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:03.685419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8b207942-b8a4-4106-925b-6d6afcc474cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:03.685852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8b207942-b8a4-4106-925b-6d6afcc474cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:03.690003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=99 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:03.715303 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8b207942-b8a4-4106-925b-6d6afcc474cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042355 Jun 10 13:11:03.715882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 370/764] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:03 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:03.740837 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6e16ff55-c345-44ca-8504-b8d47f028953 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:11:03.741374 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6e16ff55-c345-44ca-8504-b8d47f028953 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:03.747794 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e16ff55-c345-44ca-8504-b8d47f028953 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:03.748246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e16ff55-c345-44ca-8504-b8d47f028953 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:03.749548 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e16ff55-c345-44ca-8504-b8d47f028953 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:03.749984 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e16ff55-c345-44ca-8504-b8d47f028953 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:03.771969 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-6e16ff55-c345-44ca-8504-b8d47f028953 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lazy-loading 'flavor' on Instance uuid 3baaefc5-1a36-4fa6-88d9-1764e5631a32 {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:11:03.867006 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:04.074368 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6e16ff55-c345-44ca-8504-b8d47f028953 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "POST /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32/action" status: 202 len: 0 microversion: 2.1 time: 0.337527 Jun 10 13:11:04.075026 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 395/765] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Jun 10 13:11:03 2022] POST /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32/action => generated 0 bytes in 339 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jun 10 13:11:04.084309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-aa65d490-23bb-4870-9f27-1c22b361b0a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:04.084954 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-aa65d490-23bb-4870-9f27-1c22b361b0a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:04.091099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aa65d490-23bb-4870-9f27-1c22b361b0a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:04.091549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aa65d490-23bb-4870-9f27-1c22b361b0a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:04.093039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aa65d490-23bb-4870-9f27-1c22b361b0a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:04.093483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aa65d490-23bb-4870-9f27-1c22b361b0a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:04.221131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-aa65d490-23bb-4870-9f27-1c22b361b0a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:04.228643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-aa65d490-23bb-4870-9f27-1c22b361b0a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148633 Jun 10 13:11:04.233974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 371/766] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:04 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:04.726847 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-99a4ed96-4833-464c-bc47-e21407bf166e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:04.727862 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-99a4ed96-4833-464c-bc47-e21407bf166e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:04.734681 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-99a4ed96-4833-464c-bc47-e21407bf166e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:04.735127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-99a4ed96-4833-464c-bc47-e21407bf166e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:04.735891 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-99a4ed96-4833-464c-bc47-e21407bf166e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:04.736329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-99a4ed96-4833-464c-bc47-e21407bf166e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:04.765492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-99a4ed96-4833-464c-bc47-e21407bf166e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042936 Jun 10 13:11:04.766086 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 396/767] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:04 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:05.246788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d11830f5-b330-4206-8a9e-866b7f59baa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:05.247380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d11830f5-b330-4206-8a9e-866b7f59baa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:05.254320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d11830f5-b330-4206-8a9e-866b7f59baa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:05.254759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d11830f5-b330-4206-8a9e-866b7f59baa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:05.255508 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d11830f5-b330-4206-8a9e-866b7f59baa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:05.255932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d11830f5-b330-4206-8a9e-866b7f59baa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:05.387764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d11830f5-b330-4206-8a9e-866b7f59baa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:05.395369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d11830f5-b330-4206-8a9e-866b7f59baa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152618 Jun 10 13:11:05.395936 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 372/768] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:05 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:05.776436 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3ede465c-5267-4b99-95ea-bade52290de5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:05.777376 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3ede465c-5267-4b99-95ea-bade52290de5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:05.784364 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ede465c-5267-4b99-95ea-bade52290de5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:05.784837 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ede465c-5267-4b99-95ea-bade52290de5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:05.785595 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ede465c-5267-4b99-95ea-bade52290de5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:05.786029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ede465c-5267-4b99-95ea-bade52290de5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:05.814739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3ede465c-5267-4b99-95ea-bade52290de5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042192 Jun 10 13:11:05.815409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 397/769] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:05 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:06.412978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-05314921-8cd2-436d-9513-36974df6fc17 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:06.413580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-05314921-8cd2-436d-9513-36974df6fc17 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:06.419922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-05314921-8cd2-436d-9513-36974df6fc17 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:06.420360 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-05314921-8cd2-436d-9513-36974df6fc17 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:06.421128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-05314921-8cd2-436d-9513-36974df6fc17 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:06.421561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-05314921-8cd2-436d-9513-36974df6fc17 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:06.552886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-05314921-8cd2-436d-9513-36974df6fc17 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:06.561348 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-05314921-8cd2-436d-9513-36974df6fc17 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151666 Jun 10 13:11:06.562109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 373/770] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:06 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:06.825592 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2ab91c27-2b39-42fb-993f-c3cbf9887139 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:06.826502 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2ab91c27-2b39-42fb-993f-c3cbf9887139 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:06.833045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2ab91c27-2b39-42fb-993f-c3cbf9887139 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:06.833491 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2ab91c27-2b39-42fb-993f-c3cbf9887139 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:06.834250 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2ab91c27-2b39-42fb-993f-c3cbf9887139 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:06.834706 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2ab91c27-2b39-42fb-993f-c3cbf9887139 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:06.865521 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2ab91c27-2b39-42fb-993f-c3cbf9887139 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043909 Jun 10 13:11:06.866081 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 398/771] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:06 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:07.578679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-02784c22-a5bc-4fe3-8d05-afe110513ccd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:07.579195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-02784c22-a5bc-4fe3-8d05-afe110513ccd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:07.585799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-02784c22-a5bc-4fe3-8d05-afe110513ccd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:07.586249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-02784c22-a5bc-4fe3-8d05-afe110513ccd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:07.587005 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-02784c22-a5bc-4fe3-8d05-afe110513ccd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:07.587439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-02784c22-a5bc-4fe3-8d05-afe110513ccd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:07.712882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-02784c22-a5bc-4fe3-8d05-afe110513ccd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:07.720426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-02784c22-a5bc-4fe3-8d05-afe110513ccd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146272 Jun 10 13:11:07.721060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 374/772] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:07 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:07.879394 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4e5cea53-c3b9-4063-b8a9-08db8c8bd4a0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:07.880464 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4e5cea53-c3b9-4063-b8a9-08db8c8bd4a0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:07.888015 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4e5cea53-c3b9-4063-b8a9-08db8c8bd4a0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:07.888463 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4e5cea53-c3b9-4063-b8a9-08db8c8bd4a0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:07.889260 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4e5cea53-c3b9-4063-b8a9-08db8c8bd4a0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:07.889698 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4e5cea53-c3b9-4063-b8a9-08db8c8bd4a0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:07.918293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4e5cea53-c3b9-4063-b8a9-08db8c8bd4a0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045073 Jun 10 13:11:07.918842 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 399/773] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:07 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:08.739429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ec7cfa14-41a1-4e52-b1a3-e361dd27f480 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:08.740031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ec7cfa14-41a1-4e52-b1a3-e361dd27f480 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:08.746698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ec7cfa14-41a1-4e52-b1a3-e361dd27f480 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:08.747139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ec7cfa14-41a1-4e52-b1a3-e361dd27f480 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:08.747888 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ec7cfa14-41a1-4e52-b1a3-e361dd27f480 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:08.748316 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ec7cfa14-41a1-4e52-b1a3-e361dd27f480 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:08.883257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ec7cfa14-41a1-4e52-b1a3-e361dd27f480 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:08.887926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ec7cfa14-41a1-4e52-b1a3-e361dd27f480 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152496 Jun 10 13:11:08.888496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 375/774] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:08 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:08.930269 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4a99fb08-3cd1-4ede-a585-254747890cf5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:08.931202 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4a99fb08-3cd1-4ede-a585-254747890cf5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:08.937597 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4a99fb08-3cd1-4ede-a585-254747890cf5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:08.938037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4a99fb08-3cd1-4ede-a585-254747890cf5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:08.938783 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4a99fb08-3cd1-4ede-a585-254747890cf5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:08.939208 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4a99fb08-3cd1-4ede-a585-254747890cf5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:08.966486 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4a99fb08-3cd1-4ede-a585-254747890cf5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040124 Jun 10 13:11:08.967020 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 400/775] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:08 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:09.904938 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8eac3006-ace7-48ab-bfb5-03e9c4421169 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:09.905539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8eac3006-ace7-48ab-bfb5-03e9c4421169 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:09.911956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8eac3006-ace7-48ab-bfb5-03e9c4421169 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:09.912402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8eac3006-ace7-48ab-bfb5-03e9c4421169 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:09.913203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8eac3006-ace7-48ab-bfb5-03e9c4421169 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:09.913631 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8eac3006-ace7-48ab-bfb5-03e9c4421169 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:09.979134 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-79451d97-8ae5-4e59-87f6-21f2d7475b73 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:09.980156 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-79451d97-8ae5-4e59-87f6-21f2d7475b73 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:09.986905 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-79451d97-8ae5-4e59-87f6-21f2d7475b73 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:09.987419 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-79451d97-8ae5-4e59-87f6-21f2d7475b73 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:09.988652 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-79451d97-8ae5-4e59-87f6-21f2d7475b73 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:09.989169 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-79451d97-8ae5-4e59-87f6-21f2d7475b73 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:10.018794 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-79451d97-8ae5-4e59-87f6-21f2d7475b73 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043742 Jun 10 13:11:10.019365 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 401/776] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:09 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:11:10.045017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8eac3006-ace7-48ab-bfb5-03e9c4421169 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:10.052623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8eac3006-ace7-48ab-bfb5-03e9c4421169 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151755 Jun 10 13:11:10.053209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 376/777] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:09 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:11.031675 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1a83dfe4-ada6-4876-a929-df1bbe4faf51 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:11.032626 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1a83dfe4-ada6-4876-a929-df1bbe4faf51 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:11.039122 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1a83dfe4-ada6-4876-a929-df1bbe4faf51 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:11.039568 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1a83dfe4-ada6-4876-a929-df1bbe4faf51 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:11.040354 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1a83dfe4-ada6-4876-a929-df1bbe4faf51 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:11.040812 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1a83dfe4-ada6-4876-a929-df1bbe4faf51 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:11.069187 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1a83dfe4-ada6-4876-a929-df1bbe4faf51 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042616 Jun 10 13:11:11.069513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-00576121-50f5-4aed-b114-9286cb84a9d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:11.069962 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 402/778] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:11 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:11.070111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-00576121-50f5-4aed-b114-9286cb84a9d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:11.076510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00576121-50f5-4aed-b114-9286cb84a9d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:11.076989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00576121-50f5-4aed-b114-9286cb84a9d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:11.077744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00576121-50f5-4aed-b114-9286cb84a9d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:11.078208 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00576121-50f5-4aed-b114-9286cb84a9d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:11.205913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-00576121-50f5-4aed-b114-9286cb84a9d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:11.213242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-00576121-50f5-4aed-b114-9286cb84a9d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147553 Jun 10 13:11:11.213797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 377/779] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:11 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:11.698024 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4a4808bd-54c3-4ef1-a253-20f365df717c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:11.698517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4a4808bd-54c3-4ef1-a253-20f365df717c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:11.705086 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4a4808bd-54c3-4ef1-a253-20f365df717c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:11.705537 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4a4808bd-54c3-4ef1-a253-20f365df717c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:11.706302 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4a4808bd-54c3-4ef1-a253-20f365df717c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:11.706735 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4a4808bd-54c3-4ef1-a253-20f365df717c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:11.731118 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4a4808bd-54c3-4ef1-a253-20f365df717c tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7/ips" status: 200 len: 146 microversion: 2.1 time: 0.036899 Jun 10 13:11:11.731694 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 403/780] 10.222.0.25 () {58 vars in 1247 bytes} [Fri Jun 10 13:11:11 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7/ips => generated 146 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:11:12.081273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-47593473-cd8a-4356-931c-f04df103518d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:12.082190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-47593473-cd8a-4356-931c-f04df103518d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:12.088570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-47593473-cd8a-4356-931c-f04df103518d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:12.089052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-47593473-cd8a-4356-931c-f04df103518d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:12.089824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-47593473-cd8a-4356-931c-f04df103518d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:12.090259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-47593473-cd8a-4356-931c-f04df103518d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:12.120743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-47593473-cd8a-4356-931c-f04df103518d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044091 Jun 10 13:11:12.121312 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 378/781] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:12 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:12.231773 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9cbb6c73-2e2d-4523-88bb-76d1237fc057 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:12.232386 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9cbb6c73-2e2d-4523-88bb-76d1237fc057 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:12.239906 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9cbb6c73-2e2d-4523-88bb-76d1237fc057 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:12.240352 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9cbb6c73-2e2d-4523-88bb-76d1237fc057 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:12.241191 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9cbb6c73-2e2d-4523-88bb-76d1237fc057 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:12.241632 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9cbb6c73-2e2d-4523-88bb-76d1237fc057 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:12.387600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9cbb6c73-2e2d-4523-88bb-76d1237fc057 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:12.395132 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9cbb6c73-2e2d-4523-88bb-76d1237fc057 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.167726 Jun 10 13:11:12.395694 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 404/782] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:12 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:13.132790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-aca225f3-f08c-4a36-b1fc-3c0256c42f37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:13.133700 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-aca225f3-f08c-4a36-b1fc-3c0256c42f37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:13.140468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aca225f3-f08c-4a36-b1fc-3c0256c42f37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:13.140947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aca225f3-f08c-4a36-b1fc-3c0256c42f37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:13.141709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aca225f3-f08c-4a36-b1fc-3c0256c42f37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:13.142136 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aca225f3-f08c-4a36-b1fc-3c0256c42f37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:13.170353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-aca225f3-f08c-4a36-b1fc-3c0256c42f37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041535 Jun 10 13:11:13.170899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 379/783] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:13 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:13.411118 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5a1d6720-6dcb-43f1-8518-36b3abccf98e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:13.411725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5a1d6720-6dcb-43f1-8518-36b3abccf98e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:13.418253 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a1d6720-6dcb-43f1-8518-36b3abccf98e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:13.418700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a1d6720-6dcb-43f1-8518-36b3abccf98e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:13.419454 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a1d6720-6dcb-43f1-8518-36b3abccf98e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:13.419887 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a1d6720-6dcb-43f1-8518-36b3abccf98e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:13.550154 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5a1d6720-6dcb-43f1-8518-36b3abccf98e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:13.557544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5a1d6720-6dcb-43f1-8518-36b3abccf98e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150184 Jun 10 13:11:13.558109 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 405/784] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:13 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:14.182507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5eaa0bac-45ff-4a58-ad89-586fc30269da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:14.183409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5eaa0bac-45ff-4a58-ad89-586fc30269da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:14.190031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5eaa0bac-45ff-4a58-ad89-586fc30269da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:14.190421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5eaa0bac-45ff-4a58-ad89-586fc30269da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:14.191167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5eaa0bac-45ff-4a58-ad89-586fc30269da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:14.191587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5eaa0bac-45ff-4a58-ad89-586fc30269da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:14.219412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5eaa0bac-45ff-4a58-ad89-586fc30269da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040903 Jun 10 13:11:14.219948 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 380/785] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:14 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:14.575001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0f7e1876-4f8a-4bcb-93c4-4a3f72b18e82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:14.575679 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0f7e1876-4f8a-4bcb-93c4-4a3f72b18e82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:14.582749 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f7e1876-4f8a-4bcb-93c4-4a3f72b18e82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:14.583198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f7e1876-4f8a-4bcb-93c4-4a3f72b18e82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:14.583977 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f7e1876-4f8a-4bcb-93c4-4a3f72b18e82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:14.584409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f7e1876-4f8a-4bcb-93c4-4a3f72b18e82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:14.789690 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0f7e1876-4f8a-4bcb-93c4-4a3f72b18e82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:14.797198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0f7e1876-4f8a-4bcb-93c4-4a3f72b18e82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.226077 Jun 10 13:11:14.797768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 406/786] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:14 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:15.231339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bb6d3a2b-554c-423f-acc0-b125457f4a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:15.232254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bb6d3a2b-554c-423f-acc0-b125457f4a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:15.239795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb6d3a2b-554c-423f-acc0-b125457f4a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:15.240240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb6d3a2b-554c-423f-acc0-b125457f4a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:15.241031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb6d3a2b-554c-423f-acc0-b125457f4a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:15.241466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb6d3a2b-554c-423f-acc0-b125457f4a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:15.270279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bb6d3a2b-554c-423f-acc0-b125457f4a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042946 Jun 10 13:11:15.270840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 381/787] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:15 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:15.814347 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-099e6b21-fec6-40ad-8b28-2e15a048c1c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:15.814978 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-099e6b21-fec6-40ad-8b28-2e15a048c1c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:15.821482 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-099e6b21-fec6-40ad-8b28-2e15a048c1c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:15.821929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-099e6b21-fec6-40ad-8b28-2e15a048c1c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:15.822705 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-099e6b21-fec6-40ad-8b28-2e15a048c1c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:15.823134 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-099e6b21-fec6-40ad-8b28-2e15a048c1c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:15.954486 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-099e6b21-fec6-40ad-8b28-2e15a048c1c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:15.961915 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-099e6b21-fec6-40ad-8b28-2e15a048c1c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151528 Jun 10 13:11:15.962473 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 407/788] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:15 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:16.282450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7a16a7ab-7eb8-4699-87f8-0c0be0251a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:16.283400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7a16a7ab-7eb8-4699-87f8-0c0be0251a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:16.289875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a16a7ab-7eb8-4699-87f8-0c0be0251a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:16.290323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a16a7ab-7eb8-4699-87f8-0c0be0251a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:16.291093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a16a7ab-7eb8-4699-87f8-0c0be0251a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:16.291528 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a16a7ab-7eb8-4699-87f8-0c0be0251a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:16.320291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7a16a7ab-7eb8-4699-87f8-0c0be0251a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042397 Jun 10 13:11:16.320859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 382/789] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:16 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:16.980386 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-302fb687-0372-452d-bc37-9643f2b0a607 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:16.981433 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-302fb687-0372-452d-bc37-9643f2b0a607 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:16.988289 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-302fb687-0372-452d-bc37-9643f2b0a607 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:16.988738 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-302fb687-0372-452d-bc37-9643f2b0a607 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:16.989527 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-302fb687-0372-452d-bc37-9643f2b0a607 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:16.989961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-302fb687-0372-452d-bc37-9643f2b0a607 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:17.222276 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-302fb687-0372-452d-bc37-9643f2b0a607 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:17.229732 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-302fb687-0372-452d-bc37-9643f2b0a607 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.253099 Jun 10 13:11:17.235297 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 408/790] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:16 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:17.332280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-999ac528-6ea9-446a-b18e-cedbbc0f7956 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:17.333292 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-999ac528-6ea9-446a-b18e-cedbbc0f7956 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:17.335671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=44 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:17.340388 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-999ac528-6ea9-446a-b18e-cedbbc0f7956 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:17.340860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-999ac528-6ea9-446a-b18e-cedbbc0f7956 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:17.341617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-999ac528-6ea9-446a-b18e-cedbbc0f7956 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:17.342838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-999ac528-6ea9-446a-b18e-cedbbc0f7956 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:17.373880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-999ac528-6ea9-446a-b18e-cedbbc0f7956 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045824 Jun 10 13:11:17.374406 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 383/791] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:17 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:18.247612 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1ab4c05f-c6b4-4c36-bb29-627134f14d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:18.249042 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1ab4c05f-c6b4-4c36-bb29-627134f14d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:18.255228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1ab4c05f-c6b4-4c36-bb29-627134f14d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:18.255676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1ab4c05f-c6b4-4c36-bb29-627134f14d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:18.256431 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1ab4c05f-c6b4-4c36-bb29-627134f14d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:18.256898 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1ab4c05f-c6b4-4c36-bb29-627134f14d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:18.386102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b27503a9-bc17-406b-a6e9-9fdf3c0bb954 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:18.387010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b27503a9-bc17-406b-a6e9-9fdf3c0bb954 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:18.390263 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1ab4c05f-c6b4-4c36-bb29-627134f14d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:18.393911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b27503a9-bc17-406b-a6e9-9fdf3c0bb954 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:18.394349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b27503a9-bc17-406b-a6e9-9fdf3c0bb954 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:18.395130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b27503a9-bc17-406b-a6e9-9fdf3c0bb954 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:18.395563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b27503a9-bc17-406b-a6e9-9fdf3c0bb954 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:18.397670 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1ab4c05f-c6b4-4c36-bb29-627134f14d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.153871 Jun 10 13:11:18.398467 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 409/792] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:18 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:11:18.426757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b27503a9-bc17-406b-a6e9-9fdf3c0bb954 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045036 Jun 10 13:11:18.427302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 384/793] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:18 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:19.414970 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-52d9ea15-08b4-4ee6-b76a-c7493b117d6d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:19.415604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-52d9ea15-08b4-4ee6-b76a-c7493b117d6d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:19.422705 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52d9ea15-08b4-4ee6-b76a-c7493b117d6d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:19.423151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52d9ea15-08b4-4ee6-b76a-c7493b117d6d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:19.423904 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52d9ea15-08b4-4ee6-b76a-c7493b117d6d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:19.424333 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52d9ea15-08b4-4ee6-b76a-c7493b117d6d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:19.437756 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8afbce27-0f59-4fd3-9f99-e8a9cb9b7dca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:19.438772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8afbce27-0f59-4fd3-9f99-e8a9cb9b7dca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:19.445656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8afbce27-0f59-4fd3-9f99-e8a9cb9b7dca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:19.446099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8afbce27-0f59-4fd3-9f99-e8a9cb9b7dca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:19.446868 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8afbce27-0f59-4fd3-9f99-e8a9cb9b7dca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:19.447294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8afbce27-0f59-4fd3-9f99-e8a9cb9b7dca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:19.479686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8afbce27-0f59-4fd3-9f99-e8a9cb9b7dca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045873 Jun 10 13:11:19.480270 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 385/794] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:19 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:11:19.696137 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-52d9ea15-08b4-4ee6-b76a-c7493b117d6d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:19.704053 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-52d9ea15-08b4-4ee6-b76a-c7493b117d6d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.292904 Jun 10 13:11:19.704602 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 410/795] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:19 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:20.129806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-3d5f65de-e712-4447-9713-a1cae0a71b89 req-9c9f9d46-ac52-491c-8033-991bd49bf3ff service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3baaefc5-1a36-4fa6-88d9-1764e5631a32", "name": "network-vif-unplugged", "status": "completed", "tag": "72eeacbd-3288-4d28-90e4-93e208b2d17d"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:11:20.137928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-3d5f65de-e712-4447-9713-a1cae0a71b89 req-9c9f9d46-ac52-491c-8033-991bd49bf3ff service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:20.138368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-3d5f65de-e712-4447-9713-a1cae0a71b89 req-9c9f9d46-ac52-491c-8033-991bd49bf3ff service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:20.149679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-3d5f65de-e712-4447-9713-a1cae0a71b89 req-9c9f9d46-ac52-491c-8033-991bd49bf3ff service nova] Creating event network-vif-unplugged:72eeacbd-3288-4d28-90e4-93e208b2d17d for instance 3baaefc5-1a36-4fa6-88d9-1764e5631a32 on hv-cinder-maste Jun 10 13:11:20.154712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-3d5f65de-e712-4447-9713-a1cae0a71b89 req-9c9f9d46-ac52-491c-8033-991bd49bf3ff service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.083316 Jun 10 13:11:20.155290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 386/796] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:11:20 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:11:20.492381 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5b281269-c6b3-488b-adde-55e0733b542c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:20.493233 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5b281269-c6b3-488b-adde-55e0733b542c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:20.500173 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5b281269-c6b3-488b-adde-55e0733b542c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:20.500627 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5b281269-c6b3-488b-adde-55e0733b542c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:20.501403 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5b281269-c6b3-488b-adde-55e0733b542c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:20.501838 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5b281269-c6b3-488b-adde-55e0733b542c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:20.525999 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:UPDATE=1,nova_cell1:SELECT=98,nova_cell1:INSERT=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:20.530351 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5b281269-c6b3-488b-adde-55e0733b542c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042282 Jun 10 13:11:20.530884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 411/797] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:20 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:20.721968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3f600a3a-b8c5-42c5-a65c-b806c77546bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:20.722663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3f600a3a-b8c5-42c5-a65c-b806c77546bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:20.728917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f600a3a-b8c5-42c5-a65c-b806c77546bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:20.729359 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f600a3a-b8c5-42c5-a65c-b806c77546bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:20.730110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f600a3a-b8c5-42c5-a65c-b806c77546bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:20.730564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f600a3a-b8c5-42c5-a65c-b806c77546bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:20.745836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:20.860364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f600a3a-b8c5-42c5-a65c-b806c77546bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:20.868138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3f600a3a-b8c5-42c5-a65c-b806c77546bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150753 Jun 10 13:11:20.868711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 387/798] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:20 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:21.542024 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ec9af64a-362b-4bd9-a900-d1e597f68191 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:21.542936 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ec9af64a-362b-4bd9-a900-d1e597f68191 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:21.549665 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ec9af64a-362b-4bd9-a900-d1e597f68191 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:21.550109 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ec9af64a-362b-4bd9-a900-d1e597f68191 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:21.550893 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ec9af64a-362b-4bd9-a900-d1e597f68191 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:21.551324 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ec9af64a-362b-4bd9-a900-d1e597f68191 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:21.580933 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ec9af64a-362b-4bd9-a900-d1e597f68191 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043012 Jun 10 13:11:21.581690 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 412/799] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:21 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:21.885100 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-53ee940f-a90a-457a-a5b8-4061c784cadb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:21.885702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-53ee940f-a90a-457a-a5b8-4061c784cadb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:21.896390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53ee940f-a90a-457a-a5b8-4061c784cadb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:21.896910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53ee940f-a90a-457a-a5b8-4061c784cadb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:21.897703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53ee940f-a90a-457a-a5b8-4061c784cadb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:21.898188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53ee940f-a90a-457a-a5b8-4061c784cadb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:22.036656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-53ee940f-a90a-457a-a5b8-4061c784cadb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:22.044121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-53ee940f-a90a-457a-a5b8-4061c784cadb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.162853 Jun 10 13:11:22.047425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 388/800] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:21 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:22.593379 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-37fab085-110d-4db8-bc67-5d2942369a5a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:22.594360 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-37fab085-110d-4db8-bc67-5d2942369a5a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:22.601453 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37fab085-110d-4db8-bc67-5d2942369a5a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:22.601902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37fab085-110d-4db8-bc67-5d2942369a5a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:22.602660 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37fab085-110d-4db8-bc67-5d2942369a5a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:22.603125 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37fab085-110d-4db8-bc67-5d2942369a5a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:22.631556 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-37fab085-110d-4db8-bc67-5d2942369a5a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042840 Jun 10 13:11:22.632088 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 413/801] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:22 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:23.062365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a7691434-ec40-4fce-bf04-a5e24195fc84 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:23.062981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a7691434-ec40-4fce-bf04-a5e24195fc84 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:23.069488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7691434-ec40-4fce-bf04-a5e24195fc84 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:23.069940 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7691434-ec40-4fce-bf04-a5e24195fc84 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:23.070711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7691434-ec40-4fce-bf04-a5e24195fc84 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:23.071147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7691434-ec40-4fce-bf04-a5e24195fc84 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:23.200618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a7691434-ec40-4fce-bf04-a5e24195fc84 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:23.208624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a7691434-ec40-4fce-bf04-a5e24195fc84 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150690 Jun 10 13:11:23.209218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 389/802] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:23 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:23.642904 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c5aa291e-93bb-4565-ac97-7f6e8439e3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:23.643716 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c5aa291e-93bb-4565-ac97-7f6e8439e3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:23.650312 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c5aa291e-93bb-4565-ac97-7f6e8439e3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:23.650748 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c5aa291e-93bb-4565-ac97-7f6e8439e3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:23.651499 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c5aa291e-93bb-4565-ac97-7f6e8439e3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:23.651927 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c5aa291e-93bb-4565-ac97-7f6e8439e3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:23.679893 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c5aa291e-93bb-4565-ac97-7f6e8439e3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041368 Jun 10 13:11:23.680426 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 414/803] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:23 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:24.226270 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-87254231-9654-4aa7-ace6-e3073972552a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:24.226842 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-87254231-9654-4aa7-ace6-e3073972552a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:24.233431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87254231-9654-4aa7-ace6-e3073972552a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:24.233885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87254231-9654-4aa7-ace6-e3073972552a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:24.234643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87254231-9654-4aa7-ace6-e3073972552a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:24.235079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87254231-9654-4aa7-ace6-e3073972552a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:24.361977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-87254231-9654-4aa7-ace6-e3073972552a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:24.369687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-87254231-9654-4aa7-ace6-e3073972552a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147391 Jun 10 13:11:24.370252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 390/804] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:24 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:24.691505 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-98d6fe3b-bfa8-493c-a12e-8fca4b6bc68e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:24.692414 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-98d6fe3b-bfa8-493c-a12e-8fca4b6bc68e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:24.694708 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=52 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:24.699403 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-98d6fe3b-bfa8-493c-a12e-8fca4b6bc68e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:24.699845 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-98d6fe3b-bfa8-493c-a12e-8fca4b6bc68e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:24.700591 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-98d6fe3b-bfa8-493c-a12e-8fca4b6bc68e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:24.701041 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-98d6fe3b-bfa8-493c-a12e-8fca4b6bc68e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:24.729584 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-98d6fe3b-bfa8-493c-a12e-8fca4b6bc68e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042033 Jun 10 13:11:24.730123 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 415/805] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:24 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:25.386844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7bb17c37-f962-4128-aadb-f25d05d3f2ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:25.387456 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7bb17c37-f962-4128-aadb-f25d05d3f2ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:25.393933 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7bb17c37-f962-4128-aadb-f25d05d3f2ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:25.394384 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7bb17c37-f962-4128-aadb-f25d05d3f2ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:25.395142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7bb17c37-f962-4128-aadb-f25d05d3f2ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:25.395576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7bb17c37-f962-4128-aadb-f25d05d3f2ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:25.523767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7bb17c37-f962-4128-aadb-f25d05d3f2ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:25.531672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7bb17c37-f962-4128-aadb-f25d05d3f2ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148674 Jun 10 13:11:25.532237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 391/806] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:25 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:25.742015 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-68db536f-4472-4e66-b446-f79028603594 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:25.742926 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-68db536f-4472-4e66-b446-f79028603594 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:25.749380 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-68db536f-4472-4e66-b446-f79028603594 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:25.749816 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-68db536f-4472-4e66-b446-f79028603594 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:25.750561 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-68db536f-4472-4e66-b446-f79028603594 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:25.750988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-68db536f-4472-4e66-b446-f79028603594 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:25.781170 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-68db536f-4472-4e66-b446-f79028603594 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043100 Jun 10 13:11:25.781698 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 416/807] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:25 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:26.548299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-33078d89-ac9e-4308-a308-2c1cd429000f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:26.548930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-33078d89-ac9e-4308-a308-2c1cd429000f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:26.555693 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33078d89-ac9e-4308-a308-2c1cd429000f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:26.556133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33078d89-ac9e-4308-a308-2c1cd429000f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:26.556931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33078d89-ac9e-4308-a308-2c1cd429000f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:26.557355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33078d89-ac9e-4308-a308-2c1cd429000f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:26.687471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-33078d89-ac9e-4308-a308-2c1cd429000f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:26.694909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-33078d89-ac9e-4308-a308-2c1cd429000f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150954 Jun 10 13:11:26.695475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 392/808] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:26 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:26.792676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0b61c79b-3f5e-47cb-83f2-5b466d175762 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:26.793631 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0b61c79b-3f5e-47cb-83f2-5b466d175762 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:26.803935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0b61c79b-3f5e-47cb-83f2-5b466d175762 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:26.803935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0b61c79b-3f5e-47cb-83f2-5b466d175762 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:26.803935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0b61c79b-3f5e-47cb-83f2-5b466d175762 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:26.803935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0b61c79b-3f5e-47cb-83f2-5b466d175762 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:26.829995 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0b61c79b-3f5e-47cb-83f2-5b466d175762 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041259 Jun 10 13:11:26.830538 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 417/809] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:26 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:27.712046 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-56faafe5-870d-40e2-94dc-a74eca9bc7e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:27.712660 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-56faafe5-870d-40e2-94dc-a74eca9bc7e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:27.719661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-56faafe5-870d-40e2-94dc-a74eca9bc7e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:27.720112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-56faafe5-870d-40e2-94dc-a74eca9bc7e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:27.720901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-56faafe5-870d-40e2-94dc-a74eca9bc7e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:27.721353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-56faafe5-870d-40e2-94dc-a74eca9bc7e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:27.842009 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-515542ba-dd13-4e80-b8f1-e894eec8643a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:27.843423 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-515542ba-dd13-4e80-b8f1-e894eec8643a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:27.846889 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-56faafe5-870d-40e2-94dc-a74eca9bc7e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:27.850209 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-515542ba-dd13-4e80-b8f1-e894eec8643a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:27.850653 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-515542ba-dd13-4e80-b8f1-e894eec8643a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:27.851412 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-515542ba-dd13-4e80-b8f1-e894eec8643a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:27.851847 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-515542ba-dd13-4e80-b8f1-e894eec8643a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:27.854602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-56faafe5-870d-40e2-94dc-a74eca9bc7e7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146357 Jun 10 13:11:27.855438 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 393/810] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:27 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:27.881273 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-515542ba-dd13-4e80-b8f1-e894eec8643a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043534 Jun 10 13:11:27.881806 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 418/811] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:27 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:11:28.871959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-82da9e85-64e8-45a1-813f-2dc6f5f0cce1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:28.872598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-82da9e85-64e8-45a1-813f-2dc6f5f0cce1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:28.879117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-82da9e85-64e8-45a1-813f-2dc6f5f0cce1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:28.879680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-82da9e85-64e8-45a1-813f-2dc6f5f0cce1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:28.880427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-82da9e85-64e8-45a1-813f-2dc6f5f0cce1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:28.880886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-82da9e85-64e8-45a1-813f-2dc6f5f0cce1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:28.893233 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-312a5e53-5919-41ff-9b1e-6ad5b0fb8577 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:28.894170 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-312a5e53-5919-41ff-9b1e-6ad5b0fb8577 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:28.900545 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-312a5e53-5919-41ff-9b1e-6ad5b0fb8577 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:28.901038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-312a5e53-5919-41ff-9b1e-6ad5b0fb8577 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:28.901940 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-312a5e53-5919-41ff-9b1e-6ad5b0fb8577 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:28.902384 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-312a5e53-5919-41ff-9b1e-6ad5b0fb8577 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:28.940301 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-312a5e53-5919-41ff-9b1e-6ad5b0fb8577 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044219 Jun 10 13:11:28.940301 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 419/812] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:28 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:11:29.014230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-82da9e85-64e8-45a1-813f-2dc6f5f0cce1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:29.022275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-82da9e85-64e8-45a1-813f-2dc6f5f0cce1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.154127 Jun 10 13:11:29.022836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 394/813] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:28 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:29.944755 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-55d515a1-d030-48b5-853e-d541e59bc73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:29.945862 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-55d515a1-d030-48b5-853e-d541e59bc73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:29.951881 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-55d515a1-d030-48b5-853e-d541e59bc73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:29.952332 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-55d515a1-d030-48b5-853e-d541e59bc73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:29.953183 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-55d515a1-d030-48b5-853e-d541e59bc73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:29.953615 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-55d515a1-d030-48b5-853e-d541e59bc73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:29.981637 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-55d515a1-d030-48b5-853e-d541e59bc73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040891 Jun 10 13:11:29.982175 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 420/814] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:29 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:30.039850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2cea07f6-0eca-403f-bc54-15670fa5c2f5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:30.040459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2cea07f6-0eca-403f-bc54-15670fa5c2f5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:30.047559 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2cea07f6-0eca-403f-bc54-15670fa5c2f5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:30.048009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2cea07f6-0eca-403f-bc54-15670fa5c2f5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:30.048785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2cea07f6-0eca-403f-bc54-15670fa5c2f5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:30.049224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2cea07f6-0eca-403f-bc54-15670fa5c2f5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:30.175014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2cea07f6-0eca-403f-bc54-15670fa5c2f5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:30.182374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2cea07f6-0eca-403f-bc54-15670fa5c2f5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146428 Jun 10 13:11:30.182938 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 395/815] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:30 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:30.994116 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d55a5cf5-93cc-45c7-9576-bf479105cc62 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:30.995016 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d55a5cf5-93cc-45c7-9576-bf479105cc62 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:31.002012 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d55a5cf5-93cc-45c7-9576-bf479105cc62 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:31.002459 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d55a5cf5-93cc-45c7-9576-bf479105cc62 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:31.003205 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d55a5cf5-93cc-45c7-9576-bf479105cc62 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:31.003631 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d55a5cf5-93cc-45c7-9576-bf479105cc62 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:31.031281 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d55a5cf5-93cc-45c7-9576-bf479105cc62 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041115 Jun 10 13:11:31.031820 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 421/816] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:30 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:31.199211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4362490a-4c0f-476b-9ba9-06080d26d538 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:31.199817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4362490a-4c0f-476b-9ba9-06080d26d538 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:31.206053 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4362490a-4c0f-476b-9ba9-06080d26d538 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:31.206500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4362490a-4c0f-476b-9ba9-06080d26d538 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:31.207256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4362490a-4c0f-476b-9ba9-06080d26d538 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:31.207691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4362490a-4c0f-476b-9ba9-06080d26d538 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:31.335829 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4362490a-4c0f-476b-9ba9-06080d26d538 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:31.343837 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4362490a-4c0f-476b-9ba9-06080d26d538 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148561 Jun 10 13:11:31.344395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 396/817] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:31 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:32.042950 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-44fc1ecc-5cd6-409d-810e-99cfd705a794 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:32.043799 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-44fc1ecc-5cd6-409d-810e-99cfd705a794 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:32.050252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-44fc1ecc-5cd6-409d-810e-99cfd705a794 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:32.050704 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-44fc1ecc-5cd6-409d-810e-99cfd705a794 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:32.051469 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-44fc1ecc-5cd6-409d-810e-99cfd705a794 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:32.051903 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-44fc1ecc-5cd6-409d-810e-99cfd705a794 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:32.080673 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-44fc1ecc-5cd6-409d-810e-99cfd705a794 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041828 Jun 10 13:11:32.081253 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 422/818] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:32 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:32.360757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-03e5d5fb-364d-4906-887d-1b551b7accf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:32.361404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-03e5d5fb-364d-4906-887d-1b551b7accf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:32.368378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-03e5d5fb-364d-4906-887d-1b551b7accf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:32.368851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-03e5d5fb-364d-4906-887d-1b551b7accf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:32.371417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-03e5d5fb-364d-4906-887d-1b551b7accf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:32.371417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-03e5d5fb-364d-4906-887d-1b551b7accf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:32.502498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-03e5d5fb-364d-4906-887d-1b551b7accf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:32.510046 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-03e5d5fb-364d-4906-887d-1b551b7accf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.153162 Jun 10 13:11:32.510606 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 397/819] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:32 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:33.092221 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-371f5007-1b92-4b27-bf06-853c2696738f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:33.093682 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-371f5007-1b92-4b27-bf06-853c2696738f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:33.100046 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-371f5007-1b92-4b27-bf06-853c2696738f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:33.100500 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-371f5007-1b92-4b27-bf06-853c2696738f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:33.101303 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-371f5007-1b92-4b27-bf06-853c2696738f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:33.101741 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-371f5007-1b92-4b27-bf06-853c2696738f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:33.130368 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-371f5007-1b92-4b27-bf06-853c2696738f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042110 Jun 10 13:11:33.130915 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 423/820] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:33 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:33.527214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1debf9da-8e7e-45b4-9e04-1381d3a3ccc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:33.527818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1debf9da-8e7e-45b4-9e04-1381d3a3ccc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:33.535001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1debf9da-8e7e-45b4-9e04-1381d3a3ccc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:33.535440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1debf9da-8e7e-45b4-9e04-1381d3a3ccc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:33.536191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1debf9da-8e7e-45b4-9e04-1381d3a3ccc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:33.536619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1debf9da-8e7e-45b4-9e04-1381d3a3ccc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:33.663585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1debf9da-8e7e-45b4-9e04-1381d3a3ccc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:33.670955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1debf9da-8e7e-45b4-9e04-1381d3a3ccc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147590 Jun 10 13:11:33.671519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 398/821] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:33 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:34.141386 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2ee333da-cc87-4ae9-a134-1d8a375cd732 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:34.142333 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2ee333da-cc87-4ae9-a134-1d8a375cd732 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:34.148856 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2ee333da-cc87-4ae9-a134-1d8a375cd732 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:34.149307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2ee333da-cc87-4ae9-a134-1d8a375cd732 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:34.150067 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2ee333da-cc87-4ae9-a134-1d8a375cd732 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:34.150504 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2ee333da-cc87-4ae9-a134-1d8a375cd732 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:34.178541 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2ee333da-cc87-4ae9-a134-1d8a375cd732 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041166 Jun 10 13:11:34.179078 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 424/822] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:34 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:34.687999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d4bcc462-a6c6-4ac7-b96e-733cc9d3a5ec tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:34.688569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d4bcc462-a6c6-4ac7-b96e-733cc9d3a5ec tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:34.695388 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4bcc462-a6c6-4ac7-b96e-733cc9d3a5ec tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:34.695833 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4bcc462-a6c6-4ac7-b96e-733cc9d3a5ec tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:34.696592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4bcc462-a6c6-4ac7-b96e-733cc9d3a5ec tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:34.697050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4bcc462-a6c6-4ac7-b96e-733cc9d3a5ec tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:34.822973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d4bcc462-a6c6-4ac7-b96e-733cc9d3a5ec tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:34.830760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d4bcc462-a6c6-4ac7-b96e-733cc9d3a5ec tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146694 Jun 10 13:11:34.831335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 399/823] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:34 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:35.190652 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-05788088-8988-48bc-b051-f87628a1eb09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:35.191548 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-05788088-8988-48bc-b051-f87628a1eb09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:35.197951 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-05788088-8988-48bc-b051-f87628a1eb09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:35.198392 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-05788088-8988-48bc-b051-f87628a1eb09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:35.199140 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-05788088-8988-48bc-b051-f87628a1eb09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:35.199569 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-05788088-8988-48bc-b051-f87628a1eb09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:35.228188 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-05788088-8988-48bc-b051-f87628a1eb09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041993 Jun 10 13:11:35.228734 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 425/824] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:35 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:35.848210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-098c8931-a7aa-4fb3-bd78-78414f231549 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:35.848729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-098c8931-a7aa-4fb3-bd78-78414f231549 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:35.855116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-098c8931-a7aa-4fb3-bd78-78414f231549 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:35.855560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-098c8931-a7aa-4fb3-bd78-78414f231549 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:35.856308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-098c8931-a7aa-4fb3-bd78-78414f231549 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:35.856735 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-098c8931-a7aa-4fb3-bd78-78414f231549 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:35.988877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-098c8931-a7aa-4fb3-bd78-78414f231549 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:35.996395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-098c8931-a7aa-4fb3-bd78-78414f231549 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152157 Jun 10 13:11:35.996994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 400/825] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:35 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:36.239811 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4f637413-9544-4a30-bb54-00a1da63a6a0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:36.240825 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4f637413-9544-4a30-bb54-00a1da63a6a0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:36.247441 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4f637413-9544-4a30-bb54-00a1da63a6a0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:36.247895 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4f637413-9544-4a30-bb54-00a1da63a6a0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:36.248658 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4f637413-9544-4a30-bb54-00a1da63a6a0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:36.249134 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4f637413-9544-4a30-bb54-00a1da63a6a0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:36.277186 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4f637413-9544-4a30-bb54-00a1da63a6a0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041538 Jun 10 13:11:36.277731 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 426/826] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:36 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:37.013415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1cbdff16-17d2-462c-a753-44faad1bdb75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:37.014033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1cbdff16-17d2-462c-a753-44faad1bdb75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:37.021183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1cbdff16-17d2-462c-a753-44faad1bdb75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:37.021636 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1cbdff16-17d2-462c-a753-44faad1bdb75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:37.022400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1cbdff16-17d2-462c-a753-44faad1bdb75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:37.022838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1cbdff16-17d2-462c-a753-44faad1bdb75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:37.155825 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1cbdff16-17d2-462c-a753-44faad1bdb75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:37.163397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1cbdff16-17d2-462c-a753-44faad1bdb75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.153862 Jun 10 13:11:37.163965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 401/827] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:37 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:11:37.287969 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6551dc10-14ef-4802-a3c6-5c01987bd924 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:37.288906 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6551dc10-14ef-4802-a3c6-5c01987bd924 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:37.295450 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6551dc10-14ef-4802-a3c6-5c01987bd924 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:37.295892 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6551dc10-14ef-4802-a3c6-5c01987bd924 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:37.298390 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6551dc10-14ef-4802-a3c6-5c01987bd924 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:37.298390 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6551dc10-14ef-4802-a3c6-5c01987bd924 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:37.326172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6551dc10-14ef-4802-a3c6-5c01987bd924 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042195 Jun 10 13:11:37.326759 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 427/828] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:37 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:38.180476 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5cc3dd53-56a6-45d3-a4c4-3e41b5dcba09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:38.181110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5cc3dd53-56a6-45d3-a4c4-3e41b5dcba09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:38.187714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5cc3dd53-56a6-45d3-a4c4-3e41b5dcba09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:38.188294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5cc3dd53-56a6-45d3-a4c4-3e41b5dcba09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:38.189107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5cc3dd53-56a6-45d3-a4c4-3e41b5dcba09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:38.189535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5cc3dd53-56a6-45d3-a4c4-3e41b5dcba09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:38.316462 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5cc3dd53-56a6-45d3-a4c4-3e41b5dcba09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:38.324122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5cc3dd53-56a6-45d3-a4c4-3e41b5dcba09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147516 Jun 10 13:11:38.324692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 402/829] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:38 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:38.337804 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-06802599-3722-480e-8848-7d9ccc789a89 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:38.338722 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-06802599-3722-480e-8848-7d9ccc789a89 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:38.345935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-06802599-3722-480e-8848-7d9ccc789a89 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:38.346385 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-06802599-3722-480e-8848-7d9ccc789a89 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:38.347148 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-06802599-3722-480e-8848-7d9ccc789a89 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:38.347582 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-06802599-3722-480e-8848-7d9ccc789a89 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:38.364583 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:38.377196 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-06802599-3722-480e-8848-7d9ccc789a89 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043413 Jun 10 13:11:38.377763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 428/830] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:38 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:39.341484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4b85558e-9e63-44a1-a295-422c945de33e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:39.342138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4b85558e-9e63-44a1-a295-422c945de33e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:39.348571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b85558e-9e63-44a1-a295-422c945de33e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:39.349055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b85558e-9e63-44a1-a295-422c945de33e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:39.349810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b85558e-9e63-44a1-a295-422c945de33e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:39.350237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b85558e-9e63-44a1-a295-422c945de33e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:39.397201 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2801390c-6e88-43fc-8d4c-687b32a9e590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:39.398148 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2801390c-6e88-43fc-8d4c-687b32a9e590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:39.404640 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2801390c-6e88-43fc-8d4c-687b32a9e590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:39.405115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2801390c-6e88-43fc-8d4c-687b32a9e590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:39.405875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2801390c-6e88-43fc-8d4c-687b32a9e590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:39.406307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2801390c-6e88-43fc-8d4c-687b32a9e590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:39.434873 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2801390c-6e88-43fc-8d4c-687b32a9e590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041545 Jun 10 13:11:39.435443 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 429/831] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:39 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:11:39.480245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4b85558e-9e63-44a1-a295-422c945de33e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:39.488522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4b85558e-9e63-44a1-a295-422c945de33e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150903 Jun 10 13:11:39.489113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 403/832] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:39 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:40.447061 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f27eb39e-ba2f-4e6e-8aaa-9e6a2d499e9d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:40.448239 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f27eb39e-ba2f-4e6e-8aaa-9e6a2d499e9d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:40.455141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f27eb39e-ba2f-4e6e-8aaa-9e6a2d499e9d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:40.455576 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f27eb39e-ba2f-4e6e-8aaa-9e6a2d499e9d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:40.456323 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f27eb39e-ba2f-4e6e-8aaa-9e6a2d499e9d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:40.456748 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f27eb39e-ba2f-4e6e-8aaa-9e6a2d499e9d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:40.485383 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f27eb39e-ba2f-4e6e-8aaa-9e6a2d499e9d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042096 Jun 10 13:11:40.485871 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 430/833] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:40 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:40.505413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e98567b7-0ffa-4a96-ac5f-0e62b71254f4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:40.506023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e98567b7-0ffa-4a96-ac5f-0e62b71254f4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:40.512168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e98567b7-0ffa-4a96-ac5f-0e62b71254f4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:40.512551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e98567b7-0ffa-4a96-ac5f-0e62b71254f4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:40.513334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e98567b7-0ffa-4a96-ac5f-0e62b71254f4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:40.513768 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e98567b7-0ffa-4a96-ac5f-0e62b71254f4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:40.518148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:40.641022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e98567b7-0ffa-4a96-ac5f-0e62b71254f4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:40.649054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e98567b7-0ffa-4a96-ac5f-0e62b71254f4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147367 Jun 10 13:11:40.649908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 404/834] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:40 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:41.497237 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ff8ca345-28ac-433b-806a-1a30c5c02c64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:41.498145 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ff8ca345-28ac-433b-806a-1a30c5c02c64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:41.505817 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ff8ca345-28ac-433b-806a-1a30c5c02c64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:41.506264 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ff8ca345-28ac-433b-806a-1a30c5c02c64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:41.507029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ff8ca345-28ac-433b-806a-1a30c5c02c64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:41.507464 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ff8ca345-28ac-433b-806a-1a30c5c02c64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:41.536487 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ff8ca345-28ac-433b-806a-1a30c5c02c64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043176 Jun 10 13:11:41.537078 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 431/835] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:41 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:41.666117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-35ee01ec-8fa3-4a01-ac09-b811c5a1cb3b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:41.666651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-35ee01ec-8fa3-4a01-ac09-b811c5a1cb3b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:41.673767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35ee01ec-8fa3-4a01-ac09-b811c5a1cb3b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:41.674208 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35ee01ec-8fa3-4a01-ac09-b811c5a1cb3b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:41.674959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35ee01ec-8fa3-4a01-ac09-b811c5a1cb3b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:41.675388 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35ee01ec-8fa3-4a01-ac09-b811c5a1cb3b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:41.810163 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-35ee01ec-8fa3-4a01-ac09-b811c5a1cb3b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:41.817937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-35ee01ec-8fa3-4a01-ac09-b811c5a1cb3b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.155754 Jun 10 13:11:41.818497 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 405/836] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:41 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:42.548231 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1f68b3f1-6829-4139-b5f9-b8ee8d1eb96f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:42.549191 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1f68b3f1-6829-4139-b5f9-b8ee8d1eb96f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:42.555696 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1f68b3f1-6829-4139-b5f9-b8ee8d1eb96f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:42.556144 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1f68b3f1-6829-4139-b5f9-b8ee8d1eb96f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:42.556930 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1f68b3f1-6829-4139-b5f9-b8ee8d1eb96f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:42.557367 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1f68b3f1-6829-4139-b5f9-b8ee8d1eb96f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:42.585265 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1f68b3f1-6829-4139-b5f9-b8ee8d1eb96f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040999 Jun 10 13:11:42.585856 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 432/837] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:42 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:42.835258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-05ba539b-d45c-4e40-8946-9092fb23a138 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:42.835908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-05ba539b-d45c-4e40-8946-9092fb23a138 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:42.842410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-05ba539b-d45c-4e40-8946-9092fb23a138 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:42.842877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-05ba539b-d45c-4e40-8946-9092fb23a138 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:42.843626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-05ba539b-d45c-4e40-8946-9092fb23a138 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:42.844054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-05ba539b-d45c-4e40-8946-9092fb23a138 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:42.971866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-05ba539b-d45c-4e40-8946-9092fb23a138 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:42.979551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-05ba539b-d45c-4e40-8946-9092fb23a138 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148118 Jun 10 13:11:42.980118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 406/838] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:42 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:43.597457 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d8737c37-c82d-41e3-9002-87185e1bcd50 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:43.598331 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d8737c37-c82d-41e3-9002-87185e1bcd50 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:43.604664 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d8737c37-c82d-41e3-9002-87185e1bcd50 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:43.605141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d8737c37-c82d-41e3-9002-87185e1bcd50 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:43.605895 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d8737c37-c82d-41e3-9002-87185e1bcd50 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:43.606324 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d8737c37-c82d-41e3-9002-87185e1bcd50 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:43.634726 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d8737c37-c82d-41e3-9002-87185e1bcd50 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041881 Jun 10 13:11:43.635259 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 433/839] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:43 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:43.998482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-eb6fb4cf-dfe0-4a60-a080-668c0482990e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:43.999123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-eb6fb4cf-dfe0-4a60-a080-668c0482990e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:44.006211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eb6fb4cf-dfe0-4a60-a080-668c0482990e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:44.006652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eb6fb4cf-dfe0-4a60-a080-668c0482990e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:44.007489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eb6fb4cf-dfe0-4a60-a080-668c0482990e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:44.007866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eb6fb4cf-dfe0-4a60-a080-668c0482990e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:44.134492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-eb6fb4cf-dfe0-4a60-a080-668c0482990e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:44.142021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-eb6fb4cf-dfe0-4a60-a080-668c0482990e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148556 Jun 10 13:11:44.142591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 407/840] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:43 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:44.646494 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a39c80e6-7ab1-419a-8120-9bebeeba571a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:44.647324 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a39c80e6-7ab1-419a-8120-9bebeeba571a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:44.653834 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a39c80e6-7ab1-419a-8120-9bebeeba571a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:44.654272 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a39c80e6-7ab1-419a-8120-9bebeeba571a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:44.655027 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a39c80e6-7ab1-419a-8120-9bebeeba571a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:44.655457 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a39c80e6-7ab1-419a-8120-9bebeeba571a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:44.684120 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a39c80e6-7ab1-419a-8120-9bebeeba571a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041678 Jun 10 13:11:44.684723 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 434/841] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:44 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:45.158904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b212133b-5b7a-42de-9b15-775728b52b68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:45.159537 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b212133b-5b7a-42de-9b15-775728b52b68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:45.166147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b212133b-5b7a-42de-9b15-775728b52b68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:45.166589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b212133b-5b7a-42de-9b15-775728b52b68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:45.167340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b212133b-5b7a-42de-9b15-775728b52b68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:45.167765 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b212133b-5b7a-42de-9b15-775728b52b68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:45.295050 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:11:45.295266 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Fri Jun 10 13:11:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:11:45.298689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b212133b-5b7a-42de-9b15-775728b52b68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:45.306363 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b212133b-5b7a-42de-9b15-775728b52b68 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151311 Jun 10 13:11:45.306925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 408/842] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:45 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:45.696208 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-11509295-1fb6-4e23-a7de-4c3975c424ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:45.697137 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-11509295-1fb6-4e23-a7de-4c3975c424ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:45.704279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-11509295-1fb6-4e23-a7de-4c3975c424ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:45.704719 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-11509295-1fb6-4e23-a7de-4c3975c424ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:45.705492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-11509295-1fb6-4e23-a7de-4c3975c424ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:45.705916 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-11509295-1fb6-4e23-a7de-4c3975c424ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:45.733633 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-11509295-1fb6-4e23-a7de-4c3975c424ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041376 Jun 10 13:11:45.734173 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 435/843] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:45 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:46.323835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-230b0417-8c6d-4f12-9bb7-e162129028bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:46.324387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-230b0417-8c6d-4f12-9bb7-e162129028bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:46.331262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-230b0417-8c6d-4f12-9bb7-e162129028bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:46.331703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-230b0417-8c6d-4f12-9bb7-e162129028bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:46.332452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-230b0417-8c6d-4f12-9bb7-e162129028bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:46.332920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-230b0417-8c6d-4f12-9bb7-e162129028bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:46.458206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-230b0417-8c6d-4f12-9bb7-e162129028bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:46.465793 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-230b0417-8c6d-4f12-9bb7-e162129028bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145856 Jun 10 13:11:46.466367 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 409/844] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:46 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:46.745885 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-712e407e-9c8d-433b-91e6-7404c1f002e2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:46.746821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-712e407e-9c8d-433b-91e6-7404c1f002e2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:46.753736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-712e407e-9c8d-433b-91e6-7404c1f002e2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:46.754176 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-712e407e-9c8d-433b-91e6-7404c1f002e2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:46.754919 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-712e407e-9c8d-433b-91e6-7404c1f002e2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:46.755344 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-712e407e-9c8d-433b-91e6-7404c1f002e2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:46.783352 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-712e407e-9c8d-433b-91e6-7404c1f002e2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041357 Jun 10 13:11:46.783889 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 436/845] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:46 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:47.483848 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7d9ebe03-8b25-43c5-90b4-5b2d884a7090 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:47.484450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7d9ebe03-8b25-43c5-90b4-5b2d884a7090 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:47.491169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d9ebe03-8b25-43c5-90b4-5b2d884a7090 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:47.491608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d9ebe03-8b25-43c5-90b4-5b2d884a7090 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:47.492361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d9ebe03-8b25-43c5-90b4-5b2d884a7090 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:47.492809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d9ebe03-8b25-43c5-90b4-5b2d884a7090 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:47.619616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7d9ebe03-8b25-43c5-90b4-5b2d884a7090 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:47.627154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7d9ebe03-8b25-43c5-90b4-5b2d884a7090 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147131 Jun 10 13:11:47.627716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 410/846] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:47 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:47.794840 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-24f90de4-9a07-4553-965e-ef5e9cc82ff6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:47.795754 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-24f90de4-9a07-4553-965e-ef5e9cc82ff6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:47.802134 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-24f90de4-9a07-4553-965e-ef5e9cc82ff6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:47.802577 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-24f90de4-9a07-4553-965e-ef5e9cc82ff6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:47.803832 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-24f90de4-9a07-4553-965e-ef5e9cc82ff6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:47.804263 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-24f90de4-9a07-4553-965e-ef5e9cc82ff6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:47.832368 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-24f90de4-9a07-4553-965e-ef5e9cc82ff6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041445 Jun 10 13:11:47.832933 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 437/847] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:47 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:48.644483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f38acf96-477a-44a2-b5a3-dfa1248e0b0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:48.645202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f38acf96-477a-44a2-b5a3-dfa1248e0b0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:48.647514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=56 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:48.652156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f38acf96-477a-44a2-b5a3-dfa1248e0b0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:48.652600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f38acf96-477a-44a2-b5a3-dfa1248e0b0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:48.653411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f38acf96-477a-44a2-b5a3-dfa1248e0b0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:48.653815 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f38acf96-477a-44a2-b5a3-dfa1248e0b0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:48.780550 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f38acf96-477a-44a2-b5a3-dfa1248e0b0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:48.788788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f38acf96-477a-44a2-b5a3-dfa1248e0b0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148282 Jun 10 13:11:48.789358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 411/848] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:48 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:48.843734 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d9861b5f-70af-4f78-aa68-d63697aa5bd5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:48.844647 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d9861b5f-70af-4f78-aa68-d63697aa5bd5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:48.851701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d9861b5f-70af-4f78-aa68-d63697aa5bd5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:48.852148 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d9861b5f-70af-4f78-aa68-d63697aa5bd5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:48.852946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d9861b5f-70af-4f78-aa68-d63697aa5bd5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:48.853378 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d9861b5f-70af-4f78-aa68-d63697aa5bd5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:48.881853 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d9861b5f-70af-4f78-aa68-d63697aa5bd5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042083 Jun 10 13:11:48.882380 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 438/849] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:48 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:49.805694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e4fe8bbf-5265-4152-bd0b-95fa524a4938 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:49.806295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e4fe8bbf-5265-4152-bd0b-95fa524a4938 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:49.812694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4fe8bbf-5265-4152-bd0b-95fa524a4938 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:49.813166 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4fe8bbf-5265-4152-bd0b-95fa524a4938 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:49.813931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4fe8bbf-5265-4152-bd0b-95fa524a4938 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:49.814380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4fe8bbf-5265-4152-bd0b-95fa524a4938 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:49.893768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6bb278b5-39b8-4c6a-97db-6b196b521920 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:49.894739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6bb278b5-39b8-4c6a-97db-6b196b521920 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:49.901691 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6bb278b5-39b8-4c6a-97db-6b196b521920 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:49.902146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6bb278b5-39b8-4c6a-97db-6b196b521920 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:49.902922 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6bb278b5-39b8-4c6a-97db-6b196b521920 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:49.903359 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6bb278b5-39b8-4c6a-97db-6b196b521920 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:49.934137 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6bb278b5-39b8-4c6a-97db-6b196b521920 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044386 Jun 10 13:11:49.934698 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 439/850] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:49 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:11:49.946323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e4fe8bbf-5265-4152-bd0b-95fa524a4938 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:49.953986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e4fe8bbf-5265-4152-bd0b-95fa524a4938 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152080 Jun 10 13:11:49.954556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 412/851] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:49 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:50.945793 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ee035ebd-8ad5-4374-aff5-1502cd1f6913 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:50.946722 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ee035ebd-8ad5-4374-aff5-1502cd1f6913 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:50.953458 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ee035ebd-8ad5-4374-aff5-1502cd1f6913 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:50.953899 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ee035ebd-8ad5-4374-aff5-1502cd1f6913 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:50.954652 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ee035ebd-8ad5-4374-aff5-1502cd1f6913 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:50.955079 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ee035ebd-8ad5-4374-aff5-1502cd1f6913 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:50.970693 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-566de943-13e9-4033-86b2-07789be6caef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:50.971855 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-566de943-13e9-4033-86b2-07789be6caef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:50.978435 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-566de943-13e9-4033-86b2-07789be6caef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:50.978884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-566de943-13e9-4033-86b2-07789be6caef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:50.979695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-566de943-13e9-4033-86b2-07789be6caef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:50.980119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-566de943-13e9-4033-86b2-07789be6caef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:50.984700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ee035ebd-8ad5-4374-aff5-1502cd1f6913 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042823 Jun 10 13:11:50.985301 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 440/852] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:50 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:51.109947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-566de943-13e9-4033-86b2-07789be6caef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:51.117757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-566de943-13e9-4033-86b2-07789be6caef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150833 Jun 10 13:11:51.118323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 413/853] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:50 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:11:51.996280 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-25f65014-c049-4723-b746-cdd2581b7e7c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:51.997321 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-25f65014-c049-4723-b746-cdd2581b7e7c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:52.003738 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-25f65014-c049-4723-b746-cdd2581b7e7c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:52.004184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-25f65014-c049-4723-b746-cdd2581b7e7c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:52.004982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-25f65014-c049-4723-b746-cdd2581b7e7c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:52.005418 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-25f65014-c049-4723-b746-cdd2581b7e7c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:52.033702 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-25f65014-c049-4723-b746-cdd2581b7e7c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041465 Jun 10 13:11:52.034245 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 441/854] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:51 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:52.134026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6d2b7ff5-64d7-4e05-9170-46f9da01c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:52.134628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6d2b7ff5-64d7-4e05-9170-46f9da01c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:52.141119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d2b7ff5-64d7-4e05-9170-46f9da01c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:52.141564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d2b7ff5-64d7-4e05-9170-46f9da01c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:52.142462 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d2b7ff5-64d7-4e05-9170-46f9da01c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:52.142892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d2b7ff5-64d7-4e05-9170-46f9da01c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:52.270481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6d2b7ff5-64d7-4e05-9170-46f9da01c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:52.278544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6d2b7ff5-64d7-4e05-9170-46f9da01c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148477 Jun 10 13:11:52.279124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 414/855] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:52 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:53.045979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b13eb14f-dfe7-4404-a6bd-73f28abcce28 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:53.046877 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b13eb14f-dfe7-4404-a6bd-73f28abcce28 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:53.054104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b13eb14f-dfe7-4404-a6bd-73f28abcce28 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:53.059676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b13eb14f-dfe7-4404-a6bd-73f28abcce28 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:53.059676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b13eb14f-dfe7-4404-a6bd-73f28abcce28 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:53.059676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b13eb14f-dfe7-4404-a6bd-73f28abcce28 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:53.084903 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b13eb14f-dfe7-4404-a6bd-73f28abcce28 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042891 Jun 10 13:11:53.085449 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 442/856] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:53 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:53.295314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-edb39c24-8230-40d7-ab72-b835f42fd77d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:53.295908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-edb39c24-8230-40d7-ab72-b835f42fd77d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:53.302423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-edb39c24-8230-40d7-ab72-b835f42fd77d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:53.303449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-edb39c24-8230-40d7-ab72-b835f42fd77d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:53.304197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-edb39c24-8230-40d7-ab72-b835f42fd77d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:53.304704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-edb39c24-8230-40d7-ab72-b835f42fd77d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:53.434277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-edb39c24-8230-40d7-ab72-b835f42fd77d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:53.441745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-edb39c24-8230-40d7-ab72-b835f42fd77d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150256 Jun 10 13:11:53.442304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 415/857] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:53 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:54.096675 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-14697f7d-7b16-4f79-b881-65011cca4ed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:54.097617 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-14697f7d-7b16-4f79-b881-65011cca4ed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:54.104040 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14697f7d-7b16-4f79-b881-65011cca4ed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:54.104491 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14697f7d-7b16-4f79-b881-65011cca4ed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:54.105272 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14697f7d-7b16-4f79-b881-65011cca4ed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:54.105708 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14697f7d-7b16-4f79-b881-65011cca4ed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:54.133963 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-14697f7d-7b16-4f79-b881-65011cca4ed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041376 Jun 10 13:11:54.134504 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 443/858] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:54 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:54.175049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:11:54.175198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:11:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:11:54.458624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2de02bf4-0e75-4659-b033-1b392ec8cf7a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:54.463507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2de02bf4-0e75-4659-b033-1b392ec8cf7a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:54.465747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2de02bf4-0e75-4659-b033-1b392ec8cf7a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:54.466194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2de02bf4-0e75-4659-b033-1b392ec8cf7a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:54.466952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2de02bf4-0e75-4659-b033-1b392ec8cf7a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:54.467419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2de02bf4-0e75-4659-b033-1b392ec8cf7a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:54.592652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2de02bf4-0e75-4659-b033-1b392ec8cf7a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:54.600026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2de02bf4-0e75-4659-b033-1b392ec8cf7a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145272 Jun 10 13:11:54.600582 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 416/859] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:54 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:55.144727 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-05df4d0c-232c-42a0-8d9a-47a3b3b8221b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:55.145687 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-05df4d0c-232c-42a0-8d9a-47a3b3b8221b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:55.152876 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-05df4d0c-232c-42a0-8d9a-47a3b3b8221b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:55.153324 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-05df4d0c-232c-42a0-8d9a-47a3b3b8221b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:55.154118 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-05df4d0c-232c-42a0-8d9a-47a3b3b8221b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:55.154551 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-05df4d0c-232c-42a0-8d9a-47a3b3b8221b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:55.183007 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-05df4d0c-232c-42a0-8d9a-47a3b3b8221b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042205 Jun 10 13:11:55.183542 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 444/860] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:55 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:55.616691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2712761e-a33a-4635-8747-0a8fa3554820 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:55.617320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2712761e-a33a-4635-8747-0a8fa3554820 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:55.623832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2712761e-a33a-4635-8747-0a8fa3554820 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:55.624278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2712761e-a33a-4635-8747-0a8fa3554820 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:55.625090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2712761e-a33a-4635-8747-0a8fa3554820 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:55.625524 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2712761e-a33a-4635-8747-0a8fa3554820 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:55.749142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2712761e-a33a-4635-8747-0a8fa3554820 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:55.757091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2712761e-a33a-4635-8747-0a8fa3554820 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.144483 Jun 10 13:11:55.757651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 417/861] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:55 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:56.195172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c064e575-4d03-4e0c-b640-2a362080a405 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:56.196088 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c064e575-4d03-4e0c-b640-2a362080a405 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:56.198420 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=60 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:56.203131 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c064e575-4d03-4e0c-b640-2a362080a405 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:56.203577 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c064e575-4d03-4e0c-b640-2a362080a405 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:56.204336 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c064e575-4d03-4e0c-b640-2a362080a405 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:56.204787 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c064e575-4d03-4e0c-b640-2a362080a405 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:56.214079 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:56.232964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c064e575-4d03-4e0c-b640-2a362080a405 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042357 Jun 10 13:11:56.233529 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 445/862] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:56 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:56.773993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-220d3848-0ab4-43c9-b0ae-10c936c305c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:56.774596 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-220d3848-0ab4-43c9-b0ae-10c936c305c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:56.781177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-220d3848-0ab4-43c9-b0ae-10c936c305c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:56.781620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-220d3848-0ab4-43c9-b0ae-10c936c305c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:56.782373 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-220d3848-0ab4-43c9-b0ae-10c936c305c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:56.782801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-220d3848-0ab4-43c9-b0ae-10c936c305c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:56.912033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-220d3848-0ab4-43c9-b0ae-10c936c305c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:56.919383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-220d3848-0ab4-43c9-b0ae-10c936c305c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149297 Jun 10 13:11:56.919944 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 418/863] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:56 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:57.244698 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-59472ebf-767e-4743-9763-79a957816485 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:57.245645 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-59472ebf-767e-4743-9763-79a957816485 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:57.252778 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59472ebf-767e-4743-9763-79a957816485 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:57.253234 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59472ebf-767e-4743-9763-79a957816485 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:57.254007 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59472ebf-767e-4743-9763-79a957816485 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:57.254449 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59472ebf-767e-4743-9763-79a957816485 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:57.282978 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-59472ebf-767e-4743-9763-79a957816485 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043125 Jun 10 13:11:57.283529 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 446/864] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:57 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:57.936825 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-62213fa7-5fe1-468b-be40-96093f42a2e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:57.937429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-62213fa7-5fe1-468b-be40-96093f42a2e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:57.944334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62213fa7-5fe1-468b-be40-96093f42a2e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:57.944785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62213fa7-5fe1-468b-be40-96093f42a2e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:57.945548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62213fa7-5fe1-468b-be40-96093f42a2e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:57.945975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62213fa7-5fe1-468b-be40-96093f42a2e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:58.071959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-62213fa7-5fe1-468b-be40-96093f42a2e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:58.079524 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-62213fa7-5fe1-468b-be40-96093f42a2e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146439 Jun 10 13:11:58.080085 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 419/865] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:57 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:58.294987 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a2ef76c1-4d94-4115-a506-f465c0c6c0ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:58.295899 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a2ef76c1-4d94-4115-a506-f465c0c6c0ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:58.303813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a2ef76c1-4d94-4115-a506-f465c0c6c0ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:58.304258 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a2ef76c1-4d94-4115-a506-f465c0c6c0ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:58.305042 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a2ef76c1-4d94-4115-a506-f465c0c6c0ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:58.305477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a2ef76c1-4d94-4115-a506-f465c0c6c0ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:58.334094 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a2ef76c1-4d94-4115-a506-f465c0c6c0ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043038 Jun 10 13:11:58.334645 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 447/866] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:58 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:11:59.096789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8df13822-c86d-4231-903d-0ebb798e4915 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:59.097497 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8df13822-c86d-4231-903d-0ebb798e4915 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:59.104616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8df13822-c86d-4231-903d-0ebb798e4915 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:59.105091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8df13822-c86d-4231-903d-0ebb798e4915 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:59.105851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8df13822-c86d-4231-903d-0ebb798e4915 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:59.106281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8df13822-c86d-4231-903d-0ebb798e4915 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:59.234863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8df13822-c86d-4231-903d-0ebb798e4915 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:11:59.237089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:59.243190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8df13822-c86d-4231-903d-0ebb798e4915 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150240 Jun 10 13:11:59.243786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 420/867] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:11:59 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:11:59.345796 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-16a3d7e2-bfaa-48b6-8d9c-91ea87a0350b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:11:59.346703 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-16a3d7e2-bfaa-48b6-8d9c-91ea87a0350b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:11:59.353289 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-16a3d7e2-bfaa-48b6-8d9c-91ea87a0350b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:59.353669 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-16a3d7e2-bfaa-48b6-8d9c-91ea87a0350b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:59.354447 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-16a3d7e2-bfaa-48b6-8d9c-91ea87a0350b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:11:59.354872 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-16a3d7e2-bfaa-48b6-8d9c-91ea87a0350b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:11:59.384370 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-16a3d7e2-bfaa-48b6-8d9c-91ea87a0350b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042482 Jun 10 13:11:59.384945 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 448/868] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:11:59 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:00.259909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fe71e7a4-232c-45c1-9201-09b1f9b8f594 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:00.260464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fe71e7a4-232c-45c1-9201-09b1f9b8f594 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:00.267245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fe71e7a4-232c-45c1-9201-09b1f9b8f594 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:00.267688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fe71e7a4-232c-45c1-9201-09b1f9b8f594 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:00.268442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fe71e7a4-232c-45c1-9201-09b1f9b8f594 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:00.268900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fe71e7a4-232c-45c1-9201-09b1f9b8f594 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:00.395501 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-37d8541d-7a6f-4ece-b973-cf54c0836f07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:00.396448 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-37d8541d-7a6f-4ece-b973-cf54c0836f07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:00.397830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fe71e7a4-232c-45c1-9201-09b1f9b8f594 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:00.403421 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37d8541d-7a6f-4ece-b973-cf54c0836f07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:00.403862 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37d8541d-7a6f-4ece-b973-cf54c0836f07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:00.404624 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37d8541d-7a6f-4ece-b973-cf54c0836f07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:00.405081 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-37d8541d-7a6f-4ece-b973-cf54c0836f07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:00.405481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fe71e7a4-232c-45c1-9201-09b1f9b8f594 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149975 Jun 10 13:12:00.406054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 421/869] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:00 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:00.433818 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-37d8541d-7a6f-4ece-b973-cf54c0836f07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042235 Jun 10 13:12:00.434342 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 449/870] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:00 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:12:01.422766 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-95082d80-d1b4-45d3-9541-bc54247f49d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:01.423406 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-95082d80-d1b4-45d3-9541-bc54247f49d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:01.429568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95082d80-d1b4-45d3-9541-bc54247f49d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:01.430014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95082d80-d1b4-45d3-9541-bc54247f49d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:01.430767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95082d80-d1b4-45d3-9541-bc54247f49d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:01.431222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95082d80-d1b4-45d3-9541-bc54247f49d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:01.445290 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-71abe01b-7e60-4dbd-a445-e62a62a36946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:01.446294 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-71abe01b-7e60-4dbd-a445-e62a62a36946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:01.452637 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-71abe01b-7e60-4dbd-a445-e62a62a36946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:01.453108 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-71abe01b-7e60-4dbd-a445-e62a62a36946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:01.453866 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-71abe01b-7e60-4dbd-a445-e62a62a36946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:01.454290 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-71abe01b-7e60-4dbd-a445-e62a62a36946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:01.483944 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-71abe01b-7e60-4dbd-a445-e62a62a36946 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042512 Jun 10 13:12:01.484487 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 450/871] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:01 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:12:01.559786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-95082d80-d1b4-45d3-9541-bc54247f49d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:01.567254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-95082d80-d1b4-45d3-9541-bc54247f49d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148317 Jun 10 13:12:01.567811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 422/872] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:01 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:02.496342 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c953ff3c-a44b-47ae-903e-b37936bfb9e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:02.497299 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c953ff3c-a44b-47ae-903e-b37936bfb9e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:02.503693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c953ff3c-a44b-47ae-903e-b37936bfb9e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:02.504135 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c953ff3c-a44b-47ae-903e-b37936bfb9e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:02.504914 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c953ff3c-a44b-47ae-903e-b37936bfb9e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:02.505341 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c953ff3c-a44b-47ae-903e-b37936bfb9e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:02.540240 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c953ff3c-a44b-47ae-903e-b37936bfb9e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047841 Jun 10 13:12:02.540812 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 451/873] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:02 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:02.584399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fc633cb6-1a67-445e-9997-5e4b09d0e28a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:02.585027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fc633cb6-1a67-445e-9997-5e4b09d0e28a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:02.591583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fc633cb6-1a67-445e-9997-5e4b09d0e28a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:02.591975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fc633cb6-1a67-445e-9997-5e4b09d0e28a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:02.592731 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fc633cb6-1a67-445e-9997-5e4b09d0e28a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:02.593188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fc633cb6-1a67-445e-9997-5e4b09d0e28a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:02.719479 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fc633cb6-1a67-445e-9997-5e4b09d0e28a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:02.727511 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fc633cb6-1a67-445e-9997-5e4b09d0e28a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146926 Jun 10 13:12:02.728078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 423/874] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:02 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:03.551658 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-924b626c-6a5e-4c61-a688-c70206f4c5fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:03.552505 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-924b626c-6a5e-4c61-a688-c70206f4c5fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:03.559501 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-924b626c-6a5e-4c61-a688-c70206f4c5fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:03.559949 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-924b626c-6a5e-4c61-a688-c70206f4c5fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:03.560715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-924b626c-6a5e-4c61-a688-c70206f4c5fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:03.561179 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-924b626c-6a5e-4c61-a688-c70206f4c5fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:03.589951 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-924b626c-6a5e-4c61-a688-c70206f4c5fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042269 Jun 10 13:12:03.590427 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 452/875] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:03 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:03.744349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d33cd613-78e5-45ed-a452-21dbaaa82fc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:03.744978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d33cd613-78e5-45ed-a452-21dbaaa82fc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:03.751390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d33cd613-78e5-45ed-a452-21dbaaa82fc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:03.751829 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d33cd613-78e5-45ed-a452-21dbaaa82fc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:03.752575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d33cd613-78e5-45ed-a452-21dbaaa82fc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:03.753032 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d33cd613-78e5-45ed-a452-21dbaaa82fc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:03.879340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d33cd613-78e5-45ed-a452-21dbaaa82fc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:03.887403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d33cd613-78e5-45ed-a452-21dbaaa82fc8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146858 Jun 10 13:12:03.887967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 424/876] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:03 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:04.601848 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b25078ea-7ccf-438d-a98c-9e8355ea8030 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:04.602675 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b25078ea-7ccf-438d-a98c-9e8355ea8030 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:04.609091 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b25078ea-7ccf-438d-a98c-9e8355ea8030 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:04.609536 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b25078ea-7ccf-438d-a98c-9e8355ea8030 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:04.610290 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b25078ea-7ccf-438d-a98c-9e8355ea8030 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:04.610721 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b25078ea-7ccf-438d-a98c-9e8355ea8030 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:04.639077 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b25078ea-7ccf-438d-a98c-9e8355ea8030 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041800 Jun 10 13:12:04.639611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 453/877] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:04 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:12:04.904995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ce61e671-0359-4392-877b-d3c8ffd6f579 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:04.905522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ce61e671-0359-4392-877b-d3c8ffd6f579 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:04.911878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ce61e671-0359-4392-877b-d3c8ffd6f579 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:04.912993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ce61e671-0359-4392-877b-d3c8ffd6f579 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:04.913744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ce61e671-0359-4392-877b-d3c8ffd6f579 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:04.914170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ce61e671-0359-4392-877b-d3c8ffd6f579 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:05.044548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ce61e671-0359-4392-877b-d3c8ffd6f579 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:05.051992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ce61e671-0359-4392-877b-d3c8ffd6f579 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150934 Jun 10 13:12:05.052556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 425/878] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:04 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:05.650860 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-996d821f-ccf0-4a47-afdb-fbd2ad9f305c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:05.651684 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-996d821f-ccf0-4a47-afdb-fbd2ad9f305c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:05.658182 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-996d821f-ccf0-4a47-afdb-fbd2ad9f305c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:05.658624 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-996d821f-ccf0-4a47-afdb-fbd2ad9f305c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:05.659372 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-996d821f-ccf0-4a47-afdb-fbd2ad9f305c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:05.659805 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-996d821f-ccf0-4a47-afdb-fbd2ad9f305c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:05.688077 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-996d821f-ccf0-4a47-afdb-fbd2ad9f305c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041410 Jun 10 13:12:05.688610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 454/879] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:05 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:06.069645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-821b579b-4c6d-409d-b2bd-6573e940a496 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:06.070260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-821b579b-4c6d-409d-b2bd-6573e940a496 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:06.076907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-821b579b-4c6d-409d-b2bd-6573e940a496 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:06.077356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-821b579b-4c6d-409d-b2bd-6573e940a496 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:06.078121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-821b579b-4c6d-409d-b2bd-6573e940a496 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:06.078554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-821b579b-4c6d-409d-b2bd-6573e940a496 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:06.203341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-821b579b-4c6d-409d-b2bd-6573e940a496 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:06.210819 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-821b579b-4c6d-409d-b2bd-6573e940a496 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145032 Jun 10 13:12:06.211324 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 426/880] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:06 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:06.700382 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-64d0ede1-62ba-4c14-81e7-f9c8f76ee625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:06.701321 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-64d0ede1-62ba-4c14-81e7-f9c8f76ee625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:06.707773 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-64d0ede1-62ba-4c14-81e7-f9c8f76ee625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:06.708213 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-64d0ede1-62ba-4c14-81e7-f9c8f76ee625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:06.709005 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-64d0ede1-62ba-4c14-81e7-f9c8f76ee625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:06.709435 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-64d0ede1-62ba-4c14-81e7-f9c8f76ee625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:06.737273 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-64d0ede1-62ba-4c14-81e7-f9c8f76ee625 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040871 Jun 10 13:12:06.738054 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 455/881] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:06 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:07.227892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b17853e1-87c9-47c6-9e45-2196265f5506 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:07.228494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b17853e1-87c9-47c6-9e45-2196265f5506 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:07.234973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b17853e1-87c9-47c6-9e45-2196265f5506 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:07.235416 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b17853e1-87c9-47c6-9e45-2196265f5506 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:07.236165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b17853e1-87c9-47c6-9e45-2196265f5506 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:07.236592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b17853e1-87c9-47c6-9e45-2196265f5506 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:07.370387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b17853e1-87c9-47c6-9e45-2196265f5506 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:07.378331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b17853e1-87c9-47c6-9e45-2196265f5506 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.154816 Jun 10 13:12:07.379098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 427/882] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:07 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:07.749689 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1faf5246-47d4-4b65-9d35-99a007eb91e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:07.750607 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1faf5246-47d4-4b65-9d35-99a007eb91e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:07.757633 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1faf5246-47d4-4b65-9d35-99a007eb91e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:07.758080 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1faf5246-47d4-4b65-9d35-99a007eb91e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:07.758839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1faf5246-47d4-4b65-9d35-99a007eb91e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:07.759273 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1faf5246-47d4-4b65-9d35-99a007eb91e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:07.787515 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1faf5246-47d4-4b65-9d35-99a007eb91e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041787 Jun 10 13:12:07.788079 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 456/883] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:07 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:08.395337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6ef3baa9-ae44-475c-9d82-a004fe969aa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:08.395937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6ef3baa9-ae44-475c-9d82-a004fe969aa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:08.402351 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ef3baa9-ae44-475c-9d82-a004fe969aa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:08.402789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ef3baa9-ae44-475c-9d82-a004fe969aa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:08.403535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ef3baa9-ae44-475c-9d82-a004fe969aa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:08.403962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ef3baa9-ae44-475c-9d82-a004fe969aa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:08.529977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6ef3baa9-ae44-475c-9d82-a004fe969aa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:08.537925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6ef3baa9-ae44-475c-9d82-a004fe969aa1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145920 Jun 10 13:12:08.538510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 428/884] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:08 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:08.798919 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-254a5ba4-3cad-407d-9f28-74406aa8c70e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:08.799827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-254a5ba4-3cad-407d-9f28-74406aa8c70e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:08.806845 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-254a5ba4-3cad-407d-9f28-74406aa8c70e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:08.807290 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-254a5ba4-3cad-407d-9f28-74406aa8c70e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:08.808047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-254a5ba4-3cad-407d-9f28-74406aa8c70e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:08.808481 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-254a5ba4-3cad-407d-9f28-74406aa8c70e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:08.836918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-254a5ba4-3cad-407d-9f28-74406aa8c70e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042017 Jun 10 13:12:08.837620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 457/885] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:08 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:09.554712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f13eef79-4da0-4bfd-b4b3-c9da068b163f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:09.555314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f13eef79-4da0-4bfd-b4b3-c9da068b163f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:09.561473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f13eef79-4da0-4bfd-b4b3-c9da068b163f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:09.561911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f13eef79-4da0-4bfd-b4b3-c9da068b163f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:09.562666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f13eef79-4da0-4bfd-b4b3-c9da068b163f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:09.563093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f13eef79-4da0-4bfd-b4b3-c9da068b163f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:09.690517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f13eef79-4da0-4bfd-b4b3-c9da068b163f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:09.697924 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f13eef79-4da0-4bfd-b4b3-c9da068b163f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147038 Jun 10 13:12:09.698478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 429/886] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:09 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:09.847700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3ab1bd77-ee97-4a93-8dc0-10c0091c272d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:09.848616 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3ab1bd77-ee97-4a93-8dc0-10c0091c272d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:09.854752 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ab1bd77-ee97-4a93-8dc0-10c0091c272d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:09.855205 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ab1bd77-ee97-4a93-8dc0-10c0091c272d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:09.855969 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ab1bd77-ee97-4a93-8dc0-10c0091c272d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:09.856405 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ab1bd77-ee97-4a93-8dc0-10c0091c272d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:09.884784 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3ab1bd77-ee97-4a93-8dc0-10c0091c272d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041228 Jun 10 13:12:09.885326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 458/887] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:09 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:10.714269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-405b25b5-a897-4889-b2c4-59f8f90ce54f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:10.714874 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-405b25b5-a897-4889-b2c4-59f8f90ce54f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:10.721381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-405b25b5-a897-4889-b2c4-59f8f90ce54f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:10.721826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-405b25b5-a897-4889-b2c4-59f8f90ce54f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:10.722578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-405b25b5-a897-4889-b2c4-59f8f90ce54f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:10.723009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-405b25b5-a897-4889-b2c4-59f8f90ce54f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:10.850816 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-405b25b5-a897-4889-b2c4-59f8f90ce54f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:10.858189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-405b25b5-a897-4889-b2c4-59f8f90ce54f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147817 Jun 10 13:12:10.858748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 430/888] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:10 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:10.895362 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f3c0ac57-2f16-4c06-a3b5-0f60c387d446 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:10.896833 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f3c0ac57-2f16-4c06-a3b5-0f60c387d446 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:10.903304 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3c0ac57-2f16-4c06-a3b5-0f60c387d446 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:10.903747 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3c0ac57-2f16-4c06-a3b5-0f60c387d446 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:10.904497 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3c0ac57-2f16-4c06-a3b5-0f60c387d446 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:10.904951 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3c0ac57-2f16-4c06-a3b5-0f60c387d446 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:10.933414 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f3c0ac57-2f16-4c06-a3b5-0f60c387d446 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041927 Jun 10 13:12:10.933946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 459/889] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:10 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:11.875983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0b886390-e42a-4fa3-88d7-4fa9841e3bad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:11.876585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0b886390-e42a-4fa3-88d7-4fa9841e3bad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:11.883681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b886390-e42a-4fa3-88d7-4fa9841e3bad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:11.884129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b886390-e42a-4fa3-88d7-4fa9841e3bad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:11.884909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b886390-e42a-4fa3-88d7-4fa9841e3bad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:11.885344 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b886390-e42a-4fa3-88d7-4fa9841e3bad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:11.945647 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-51b274bd-73e0-4201-a6f0-7b00c3a34b3c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:11.946679 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-51b274bd-73e0-4201-a6f0-7b00c3a34b3c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:11.952948 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-51b274bd-73e0-4201-a6f0-7b00c3a34b3c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:11.953396 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-51b274bd-73e0-4201-a6f0-7b00c3a34b3c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:11.954152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-51b274bd-73e0-4201-a6f0-7b00c3a34b3c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:11.954583 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-51b274bd-73e0-4201-a6f0-7b00c3a34b3c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:11.987148 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-51b274bd-73e0-4201-a6f0-7b00c3a34b3c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046045 Jun 10 13:12:11.987975 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 460/890] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:11 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:12:12.021490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0b886390-e42a-4fa3-88d7-4fa9841e3bad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:12.029023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0b886390-e42a-4fa3-88d7-4fa9841e3bad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.156909 Jun 10 13:12:12.029587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 431/891] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:11 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:12.999732 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9e87617a-1575-43b2-930f-57103fa18d05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:13.000651 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9e87617a-1575-43b2-930f-57103fa18d05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:13.007684 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9e87617a-1575-43b2-930f-57103fa18d05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:13.008138 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9e87617a-1575-43b2-930f-57103fa18d05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:13.008923 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9e87617a-1575-43b2-930f-57103fa18d05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:13.009791 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9e87617a-1575-43b2-930f-57103fa18d05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:13.033855 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:13.038232 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9e87617a-1575-43b2-930f-57103fa18d05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042490 Jun 10 13:12:13.038773 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 461/892] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:12 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:13.046897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1ec7fd9d-693f-4735-914f-89a21895915c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:13.047504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1ec7fd9d-693f-4735-914f-89a21895915c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:13.054034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ec7fd9d-693f-4735-914f-89a21895915c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:13.055645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ec7fd9d-693f-4735-914f-89a21895915c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:13.055645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ec7fd9d-693f-4735-914f-89a21895915c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:13.055645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ec7fd9d-693f-4735-914f-89a21895915c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:13.367311 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1ec7fd9d-693f-4735-914f-89a21895915c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:13.374983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1ec7fd9d-693f-4735-914f-89a21895915c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.332295 Jun 10 13:12:13.375544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 432/893] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:13 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:14.050244 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e67fb670-4d91-4bfb-8c30-2b76d70ad671 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:14.051162 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e67fb670-4d91-4bfb-8c30-2b76d70ad671 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:14.058261 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e67fb670-4d91-4bfb-8c30-2b76d70ad671 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:14.062167 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e67fb670-4d91-4bfb-8c30-2b76d70ad671 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:14.062167 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e67fb670-4d91-4bfb-8c30-2b76d70ad671 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:14.062167 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e67fb670-4d91-4bfb-8c30-2b76d70ad671 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:14.091954 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e67fb670-4d91-4bfb-8c30-2b76d70ad671 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046066 Jun 10 13:12:14.092515 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 462/894] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:14 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:14.392501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c72f5fce-de17-4132-b91e-d38f76a2c841 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:14.393144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c72f5fce-de17-4132-b91e-d38f76a2c841 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:14.400261 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c72f5fce-de17-4132-b91e-d38f76a2c841 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:14.400707 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c72f5fce-de17-4132-b91e-d38f76a2c841 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:14.401490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c72f5fce-de17-4132-b91e-d38f76a2c841 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:14.401923 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c72f5fce-de17-4132-b91e-d38f76a2c841 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:14.527365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c72f5fce-de17-4132-b91e-d38f76a2c841 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:14.534756 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c72f5fce-de17-4132-b91e-d38f76a2c841 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146097 Jun 10 13:12:14.535319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 433/895] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:14 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:15.103151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-42a68145-2cd9-4500-a368-d04e6ce66d06 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:15.104060 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-42a68145-2cd9-4500-a368-d04e6ce66d06 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:15.110499 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-42a68145-2cd9-4500-a368-d04e6ce66d06 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:15.110946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-42a68145-2cd9-4500-a368-d04e6ce66d06 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:15.111713 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-42a68145-2cd9-4500-a368-d04e6ce66d06 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:15.112144 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-42a68145-2cd9-4500-a368-d04e6ce66d06 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:15.140896 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-42a68145-2cd9-4500-a368-d04e6ce66d06 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041733 Jun 10 13:12:15.141444 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 463/896] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:15 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:15.552140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5e57c9fa-35ab-44f2-a769-9fc5f74e1ede tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:15.553298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5e57c9fa-35ab-44f2-a769-9fc5f74e1ede tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:15.559689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e57c9fa-35ab-44f2-a769-9fc5f74e1ede tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:15.560131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e57c9fa-35ab-44f2-a769-9fc5f74e1ede tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:15.560900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e57c9fa-35ab-44f2-a769-9fc5f74e1ede tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:15.561333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e57c9fa-35ab-44f2-a769-9fc5f74e1ede tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:15.697782 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5e57c9fa-35ab-44f2-a769-9fc5f74e1ede tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:15.705778 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5e57c9fa-35ab-44f2-a769-9fc5f74e1ede tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.157369 Jun 10 13:12:15.706393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 434/897] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:15 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:16.153334 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ead7a0e3-3df7-4f6b-8bb0-016676ee4130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:16.154251 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ead7a0e3-3df7-4f6b-8bb0-016676ee4130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:16.160709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ead7a0e3-3df7-4f6b-8bb0-016676ee4130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:16.161188 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ead7a0e3-3df7-4f6b-8bb0-016676ee4130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:16.162605 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ead7a0e3-3df7-4f6b-8bb0-016676ee4130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:16.163044 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ead7a0e3-3df7-4f6b-8bb0-016676ee4130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:16.190739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ead7a0e3-3df7-4f6b-8bb0-016676ee4130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041681 Jun 10 13:12:16.191297 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 464/898] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:16 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:16.723088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-40e27bbc-4945-4d75-b475-0c595e1f88ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:16.730139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-40e27bbc-4945-4d75-b475-0c595e1f88ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:16.730350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-40e27bbc-4945-4d75-b475-0c595e1f88ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:16.730795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-40e27bbc-4945-4d75-b475-0c595e1f88ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:16.731546 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-40e27bbc-4945-4d75-b475-0c595e1f88ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:16.731977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-40e27bbc-4945-4d75-b475-0c595e1f88ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:16.859795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-40e27bbc-4945-4d75-b475-0c595e1f88ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:16.868181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-40e27bbc-4945-4d75-b475-0c595e1f88ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148924 Jun 10 13:12:16.868742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 435/899] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:16 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:17.202326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-192ca690-14b0-4550-bdb8-96f254f1e85e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:17.203276 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-192ca690-14b0-4550-bdb8-96f254f1e85e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:17.209848 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-192ca690-14b0-4550-bdb8-96f254f1e85e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:17.217717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-192ca690-14b0-4550-bdb8-96f254f1e85e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:17.217717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-192ca690-14b0-4550-bdb8-96f254f1e85e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:17.217717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-192ca690-14b0-4550-bdb8-96f254f1e85e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:17.241918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-192ca690-14b0-4550-bdb8-96f254f1e85e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043595 Jun 10 13:12:17.242478 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 465/900] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:17 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:17.885536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f872b95c-0926-446e-bf7b-5969d4d3efad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:17.886173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f872b95c-0926-446e-bf7b-5969d4d3efad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:17.892539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f872b95c-0926-446e-bf7b-5969d4d3efad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:17.893008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f872b95c-0926-446e-bf7b-5969d4d3efad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:17.893759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f872b95c-0926-446e-bf7b-5969d4d3efad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:17.894186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f872b95c-0926-446e-bf7b-5969d4d3efad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:18.018081 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f872b95c-0926-446e-bf7b-5969d4d3efad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:18.025643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f872b95c-0926-446e-bf7b-5969d4d3efad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.143916 Jun 10 13:12:18.026206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 436/901] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:17 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:12:18.259975 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c28c039a-9bcc-4fb8-a615-f9d5fae6e8cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:18.260912 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c28c039a-9bcc-4fb8-a615-f9d5fae6e8cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:18.267517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c28c039a-9bcc-4fb8-a615-f9d5fae6e8cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:18.267959 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c28c039a-9bcc-4fb8-a615-f9d5fae6e8cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:18.268707 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c28c039a-9bcc-4fb8-a615-f9d5fae6e8cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:18.269159 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c28c039a-9bcc-4fb8-a615-f9d5fae6e8cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:18.297541 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c28c039a-9bcc-4fb8-a615-f9d5fae6e8cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041834 Jun 10 13:12:18.298088 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 466/902] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:18 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:19.043011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-01905820-ede9-44d5-b93e-8f55ed605974 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:19.043613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-01905820-ede9-44d5-b93e-8f55ed605974 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:19.045925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=52 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:19.050629 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01905820-ede9-44d5-b93e-8f55ed605974 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:19.051073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01905820-ede9-44d5-b93e-8f55ed605974 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:19.051831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01905820-ede9-44d5-b93e-8f55ed605974 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:19.052267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01905820-ede9-44d5-b93e-8f55ed605974 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:19.067477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:19.182857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-01905820-ede9-44d5-b93e-8f55ed605974 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:19.190385 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-01905820-ede9-44d5-b93e-8f55ed605974 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151261 Jun 10 13:12:19.190951 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 437/903] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:19 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:19.308704 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4c141a49-da08-4a74-8ff2-e87923745dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:19.309568 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4c141a49-da08-4a74-8ff2-e87923745dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:19.316029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4c141a49-da08-4a74-8ff2-e87923745dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:19.316475 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4c141a49-da08-4a74-8ff2-e87923745dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:19.317266 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4c141a49-da08-4a74-8ff2-e87923745dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:19.317699 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4c141a49-da08-4a74-8ff2-e87923745dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:19.347016 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4c141a49-da08-4a74-8ff2-e87923745dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042271 Jun 10 13:12:19.347568 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 467/904] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:19 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:20.207205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-614f71d5-02fe-4847-9b3e-cb48293e9cc7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:20.207802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-614f71d5-02fe-4847-9b3e-cb48293e9cc7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:20.214545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-614f71d5-02fe-4847-9b3e-cb48293e9cc7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:20.214988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-614f71d5-02fe-4847-9b3e-cb48293e9cc7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:20.215796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-614f71d5-02fe-4847-9b3e-cb48293e9cc7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:20.216173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-614f71d5-02fe-4847-9b3e-cb48293e9cc7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:20.345117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-614f71d5-02fe-4847-9b3e-cb48293e9cc7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:20.352573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-614f71d5-02fe-4847-9b3e-cb48293e9cc7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149190 Jun 10 13:12:20.353171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 438/905] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:20 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:20.359392 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3c033349-74e2-45c9-ac70-1dfdb37468c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:20.360394 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3c033349-74e2-45c9-ac70-1dfdb37468c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:20.367404 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c033349-74e2-45c9-ac70-1dfdb37468c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:20.367850 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c033349-74e2-45c9-ac70-1dfdb37468c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:20.368597 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c033349-74e2-45c9-ac70-1dfdb37468c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:20.369047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c033349-74e2-45c9-ac70-1dfdb37468c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:20.397613 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3c033349-74e2-45c9-ac70-1dfdb37468c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042087 Jun 10 13:12:20.398146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 468/906] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:20 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:21.370020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-86a77b25-ae03-4536-b455-9756ffb56a49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:21.370625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-86a77b25-ae03-4536-b455-9756ffb56a49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:21.377192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-86a77b25-ae03-4536-b455-9756ffb56a49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:21.377634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-86a77b25-ae03-4536-b455-9756ffb56a49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:21.378386 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-86a77b25-ae03-4536-b455-9756ffb56a49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:21.378818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-86a77b25-ae03-4536-b455-9756ffb56a49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:21.409168 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-10895c12-7f9a-4b70-a763-3c9282d401cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:21.410194 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-10895c12-7f9a-4b70-a763-3c9282d401cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:21.416687 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-10895c12-7f9a-4b70-a763-3c9282d401cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:21.417300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-10895c12-7f9a-4b70-a763-3c9282d401cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:21.418070 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-10895c12-7f9a-4b70-a763-3c9282d401cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:21.418502 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-10895c12-7f9a-4b70-a763-3c9282d401cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:21.448016 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-10895c12-7f9a-4b70-a763-3c9282d401cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042781 Jun 10 13:12:21.448570 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 469/907] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:21 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:12:21.507960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-86a77b25-ae03-4536-b455-9756ffb56a49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:21.515355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-86a77b25-ae03-4536-b455-9756ffb56a49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149279 Jun 10 13:12:21.515917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 439/908] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:21 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:22.460045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0530e5ca-c95e-45a4-a940-e3c2c0ff026b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:22.461003 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0530e5ca-c95e-45a4-a940-e3c2c0ff026b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:22.467429 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0530e5ca-c95e-45a4-a940-e3c2c0ff026b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:22.467880 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0530e5ca-c95e-45a4-a940-e3c2c0ff026b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:22.468677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0530e5ca-c95e-45a4-a940-e3c2c0ff026b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:22.469138 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0530e5ca-c95e-45a4-a940-e3c2c0ff026b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:22.498566 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0530e5ca-c95e-45a4-a940-e3c2c0ff026b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042981 Jun 10 13:12:22.499106 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 470/909] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:22 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:22.531115 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c31a7636-b8ef-4c34-b382-6b49ec10691d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:22.531714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c31a7636-b8ef-4c34-b382-6b49ec10691d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:22.538145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c31a7636-b8ef-4c34-b382-6b49ec10691d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:22.538583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c31a7636-b8ef-4c34-b382-6b49ec10691d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:22.539332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c31a7636-b8ef-4c34-b382-6b49ec10691d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:22.539786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c31a7636-b8ef-4c34-b382-6b49ec10691d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:22.668321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c31a7636-b8ef-4c34-b382-6b49ec10691d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:22.675763 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c31a7636-b8ef-4c34-b382-6b49ec10691d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148404 Jun 10 13:12:22.676327 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 440/910] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:22 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:23.509362 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4ca662b9-781e-43c0-b720-d57a07357421 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:23.510278 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4ca662b9-781e-43c0-b720-d57a07357421 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:23.516752 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4ca662b9-781e-43c0-b720-d57a07357421 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:23.517228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4ca662b9-781e-43c0-b720-d57a07357421 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:23.517985 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4ca662b9-781e-43c0-b720-d57a07357421 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:23.518417 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4ca662b9-781e-43c0-b720-d57a07357421 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:23.546351 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4ca662b9-781e-43c0-b720-d57a07357421 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040966 Jun 10 13:12:23.547102 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 471/911] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:23 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:23.692801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-61ff9723-977e-4008-a45b-28020b702805 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:23.693411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-61ff9723-977e-4008-a45b-28020b702805 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:23.699773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61ff9723-977e-4008-a45b-28020b702805 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:23.700223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61ff9723-977e-4008-a45b-28020b702805 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:23.701008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61ff9723-977e-4008-a45b-28020b702805 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:23.701441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61ff9723-977e-4008-a45b-28020b702805 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:23.831710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-61ff9723-977e-4008-a45b-28020b702805 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:23.839747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-61ff9723-977e-4008-a45b-28020b702805 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150897 Jun 10 13:12:23.840310 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 441/912] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:23 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:24.558378 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7595182d-042c-423b-8029-d2d26a6838c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:24.559283 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7595182d-042c-423b-8029-d2d26a6838c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:24.565691 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7595182d-042c-423b-8029-d2d26a6838c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:24.566133 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7595182d-042c-423b-8029-d2d26a6838c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:24.566883 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7595182d-042c-423b-8029-d2d26a6838c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:24.567318 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7595182d-042c-423b-8029-d2d26a6838c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:24.596877 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7595182d-042c-423b-8029-d2d26a6838c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042351 Jun 10 13:12:24.597330 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 472/913] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:24 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:24.856637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0eea7b0e-cf2e-4e5e-82d2-b05f9e003e94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:24.857195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0eea7b0e-cf2e-4e5e-82d2-b05f9e003e94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:24.863811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0eea7b0e-cf2e-4e5e-82d2-b05f9e003e94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:24.864251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0eea7b0e-cf2e-4e5e-82d2-b05f9e003e94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:24.865593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0eea7b0e-cf2e-4e5e-82d2-b05f9e003e94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:24.866019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0eea7b0e-cf2e-4e5e-82d2-b05f9e003e94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:25.138120 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0eea7b0e-cf2e-4e5e-82d2-b05f9e003e94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:25.145500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0eea7b0e-cf2e-4e5e-82d2-b05f9e003e94 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.292745 Jun 10 13:12:25.146562 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 442/914] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:24 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:25.608428 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4b553c3d-1c6b-4f27-9c68-0a36c3bcc31a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:25.609372 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4b553c3d-1c6b-4f27-9c68-0a36c3bcc31a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:25.616112 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b553c3d-1c6b-4f27-9c68-0a36c3bcc31a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:25.616554 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b553c3d-1c6b-4f27-9c68-0a36c3bcc31a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:25.617349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b553c3d-1c6b-4f27-9c68-0a36c3bcc31a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:25.617780 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b553c3d-1c6b-4f27-9c68-0a36c3bcc31a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:25.645800 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4b553c3d-1c6b-4f27-9c68-0a36c3bcc31a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041391 Jun 10 13:12:25.646961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 473/915] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:25 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:26.162903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e7e6902c-98a8-4f25-ab6d-b1d67d4fde73 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:26.163502 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e7e6902c-98a8-4f25-ab6d-b1d67d4fde73 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:26.169899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7e6902c-98a8-4f25-ab6d-b1d67d4fde73 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:26.170342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7e6902c-98a8-4f25-ab6d-b1d67d4fde73 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:26.171085 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7e6902c-98a8-4f25-ab6d-b1d67d4fde73 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:26.171507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7e6902c-98a8-4f25-ab6d-b1d67d4fde73 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:26.295989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e7e6902c-98a8-4f25-ab6d-b1d67d4fde73 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:26.304098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e7e6902c-98a8-4f25-ab6d-b1d67d4fde73 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145030 Jun 10 13:12:26.304652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 443/916] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:26 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:26.658073 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d10c9d60-d6cc-446e-86f6-c8f9dee619f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:26.658910 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d10c9d60-d6cc-446e-86f6-c8f9dee619f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:26.666118 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d10c9d60-d6cc-446e-86f6-c8f9dee619f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:26.666567 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d10c9d60-d6cc-446e-86f6-c8f9dee619f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:26.667322 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d10c9d60-d6cc-446e-86f6-c8f9dee619f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:26.667757 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d10c9d60-d6cc-446e-86f6-c8f9dee619f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:26.695654 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d10c9d60-d6cc-446e-86f6-c8f9dee619f2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041581 Jun 10 13:12:26.696195 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 474/917] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:26 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:12:27.321136 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fe53b163-032c-4a39-9b7b-151f73f11a10 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:27.321739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fe53b163-032c-4a39-9b7b-151f73f11a10 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:27.328394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fe53b163-032c-4a39-9b7b-151f73f11a10 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:27.328866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fe53b163-032c-4a39-9b7b-151f73f11a10 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:27.329622 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fe53b163-032c-4a39-9b7b-151f73f11a10 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:27.330052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fe53b163-032c-4a39-9b7b-151f73f11a10 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:27.459848 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fe53b163-032c-4a39-9b7b-151f73f11a10 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:27.467301 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fe53b163-032c-4a39-9b7b-151f73f11a10 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150593 Jun 10 13:12:27.467847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 444/918] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:27 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:27.707357 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-14380ff6-acad-422e-a707-88e18d51f33e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:27.708265 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-14380ff6-acad-422e-a707-88e18d51f33e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:27.710534 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=60 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:27.715192 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14380ff6-acad-422e-a707-88e18d51f33e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:27.715635 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14380ff6-acad-422e-a707-88e18d51f33e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:27.716386 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14380ff6-acad-422e-a707-88e18d51f33e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:27.716846 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14380ff6-acad-422e-a707-88e18d51f33e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:27.746430 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-14380ff6-acad-422e-a707-88e18d51f33e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043089 Jun 10 13:12:27.746977 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 475/919] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:27 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:28.484064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0c04f133-79db-4865-b98a-d292658a56e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:28.484667 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0c04f133-79db-4865-b98a-d292658a56e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:28.491364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c04f133-79db-4865-b98a-d292658a56e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:28.491806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c04f133-79db-4865-b98a-d292658a56e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:28.492551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c04f133-79db-4865-b98a-d292658a56e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:28.493003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c04f133-79db-4865-b98a-d292658a56e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:28.627930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0c04f133-79db-4865-b98a-d292658a56e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:28.635912 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0c04f133-79db-4865-b98a-d292658a56e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.155672 Jun 10 13:12:28.636468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 445/920] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:28 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:28.758829 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-782d274c-578b-4349-9764-6b959887aea3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:28.759743 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-782d274c-578b-4349-9764-6b959887aea3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:28.766727 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-782d274c-578b-4349-9764-6b959887aea3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:28.767174 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-782d274c-578b-4349-9764-6b959887aea3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:28.767931 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-782d274c-578b-4349-9764-6b959887aea3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:28.768361 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-782d274c-578b-4349-9764-6b959887aea3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:28.797021 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-782d274c-578b-4349-9764-6b959887aea3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042688 Jun 10 13:12:28.797573 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 476/921] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:28 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:29.653361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c5115fd9-6aeb-46a8-b487-be12f7233815 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:29.653894 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c5115fd9-6aeb-46a8-b487-be12f7233815 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:29.660299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5115fd9-6aeb-46a8-b487-be12f7233815 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:29.660736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5115fd9-6aeb-46a8-b487-be12f7233815 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:29.661515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5115fd9-6aeb-46a8-b487-be12f7233815 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:29.662485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5115fd9-6aeb-46a8-b487-be12f7233815 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:29.792692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c5115fd9-6aeb-46a8-b487-be12f7233815 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:29.800265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c5115fd9-6aeb-46a8-b487-be12f7233815 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150778 Jun 10 13:12:29.800833 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 446/922] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:29 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:29.807417 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-221b66ad-81ed-408d-9e4f-b1ef6097c5ba tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:29.808324 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-221b66ad-81ed-408d-9e4f-b1ef6097c5ba tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:29.814716 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-221b66ad-81ed-408d-9e4f-b1ef6097c5ba tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:29.815158 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-221b66ad-81ed-408d-9e4f-b1ef6097c5ba tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:29.815913 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-221b66ad-81ed-408d-9e4f-b1ef6097c5ba tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:29.816342 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-221b66ad-81ed-408d-9e4f-b1ef6097c5ba tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:29.844475 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-221b66ad-81ed-408d-9e4f-b1ef6097c5ba tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040953 Jun 10 13:12:29.845046 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 477/923] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:29 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:30.817196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bb7723a8-b70c-4ca0-9584-3450c8f03545 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:30.817876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bb7723a8-b70c-4ca0-9584-3450c8f03545 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:30.824242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb7723a8-b70c-4ca0-9584-3450c8f03545 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:30.824691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb7723a8-b70c-4ca0-9584-3450c8f03545 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:30.825468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb7723a8-b70c-4ca0-9584-3450c8f03545 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:30.825899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb7723a8-b70c-4ca0-9584-3450c8f03545 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:30.855987 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-79ac5cfc-c227-4b5d-9121-650bb3111d1d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:30.856922 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-79ac5cfc-c227-4b5d-9121-650bb3111d1d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:30.863751 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-79ac5cfc-c227-4b5d-9121-650bb3111d1d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:30.864193 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-79ac5cfc-c227-4b5d-9121-650bb3111d1d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:30.865553 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-79ac5cfc-c227-4b5d-9121-650bb3111d1d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:30.866030 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-79ac5cfc-c227-4b5d-9121-650bb3111d1d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:30.882416 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:30.894762 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-79ac5cfc-c227-4b5d-9121-650bb3111d1d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042668 Jun 10 13:12:30.895312 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 478/924] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:30 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:12:30.952940 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bb7723a8-b70c-4ca0-9584-3450c8f03545 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:30.960227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bb7723a8-b70c-4ca0-9584-3450c8f03545 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146891 Jun 10 13:12:30.960798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 447/925] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:30 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:31.906715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-62a933f3-733c-4b5b-90d9-b7f7035c3679 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:31.907630 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-62a933f3-733c-4b5b-90d9-b7f7035c3679 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:31.914828 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62a933f3-733c-4b5b-90d9-b7f7035c3679 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:31.915267 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62a933f3-733c-4b5b-90d9-b7f7035c3679 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:31.916019 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62a933f3-733c-4b5b-90d9-b7f7035c3679 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:31.916447 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62a933f3-733c-4b5b-90d9-b7f7035c3679 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:31.943994 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-62a933f3-733c-4b5b-90d9-b7f7035c3679 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041602 Jun 10 13:12:31.944532 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 479/926] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:31 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:32.082617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7742c489-6b77-48b2-8747-4b7b348127b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:32.083248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7742c489-6b77-48b2-8747-4b7b348127b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:32.089651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7742c489-6b77-48b2-8747-4b7b348127b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:32.090093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7742c489-6b77-48b2-8747-4b7b348127b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:32.090845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7742c489-6b77-48b2-8747-4b7b348127b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:32.091304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7742c489-6b77-48b2-8747-4b7b348127b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:32.220450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7742c489-6b77-48b2-8747-4b7b348127b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:32.228469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7742c489-6b77-48b2-8747-4b7b348127b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.254969 Jun 10 13:12:32.229058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 448/927] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:31 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:32.954746 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-eadbd7d7-2049-4f16-8169-f98d110ff9fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:32.955655 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-eadbd7d7-2049-4f16-8169-f98d110ff9fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:32.962354 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eadbd7d7-2049-4f16-8169-f98d110ff9fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:32.962803 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eadbd7d7-2049-4f16-8169-f98d110ff9fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:32.963556 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eadbd7d7-2049-4f16-8169-f98d110ff9fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:32.963994 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eadbd7d7-2049-4f16-8169-f98d110ff9fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:32.994185 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-eadbd7d7-2049-4f16-8169-f98d110ff9fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043440 Jun 10 13:12:32.994729 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 480/928] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:32 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:33.246191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e8fc4fb9-ea02-4ec0-8d81-ac4a6a255cac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:33.246711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e8fc4fb9-ea02-4ec0-8d81-ac4a6a255cac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:33.253131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8fc4fb9-ea02-4ec0-8d81-ac4a6a255cac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:33.253579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8fc4fb9-ea02-4ec0-8d81-ac4a6a255cac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:33.254335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8fc4fb9-ea02-4ec0-8d81-ac4a6a255cac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:33.254767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8fc4fb9-ea02-4ec0-8d81-ac4a6a255cac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:33.387514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e8fc4fb9-ea02-4ec0-8d81-ac4a6a255cac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:33.395003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e8fc4fb9-ea02-4ec0-8d81-ac4a6a255cac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152774 Jun 10 13:12:33.395805 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 449/929] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:33 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:34.005863 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c8ea823b-3fff-46a3-b709-0791cf5de276 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:34.006767 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c8ea823b-3fff-46a3-b709-0791cf5de276 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:34.013155 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c8ea823b-3fff-46a3-b709-0791cf5de276 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:34.013599 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c8ea823b-3fff-46a3-b709-0791cf5de276 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:34.014374 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c8ea823b-3fff-46a3-b709-0791cf5de276 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:34.014807 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c8ea823b-3fff-46a3-b709-0791cf5de276 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:34.042909 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c8ea823b-3fff-46a3-b709-0791cf5de276 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041077 Jun 10 13:12:34.043447 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 481/930] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:34 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:34.413318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e111cea1-d29b-4ac4-8b51-63174c441e7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:34.413922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e111cea1-d29b-4ac4-8b51-63174c441e7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:34.420390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e111cea1-d29b-4ac4-8b51-63174c441e7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:34.420878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e111cea1-d29b-4ac4-8b51-63174c441e7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:34.421634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e111cea1-d29b-4ac4-8b51-63174c441e7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:34.422063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e111cea1-d29b-4ac4-8b51-63174c441e7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:34.546225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e111cea1-d29b-4ac4-8b51-63174c441e7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:34.553647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e111cea1-d29b-4ac4-8b51-63174c441e7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.144851 Jun 10 13:12:34.554200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 450/931] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:34 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:35.054965 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7aeb1640-125e-4a78-8d92-cc75715bf296 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:35.055865 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7aeb1640-125e-4a78-8d92-cc75715bf296 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:35.062254 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7aeb1640-125e-4a78-8d92-cc75715bf296 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:35.062695 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7aeb1640-125e-4a78-8d92-cc75715bf296 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:35.063440 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7aeb1640-125e-4a78-8d92-cc75715bf296 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:35.063877 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7aeb1640-125e-4a78-8d92-cc75715bf296 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:35.092320 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7aeb1640-125e-4a78-8d92-cc75715bf296 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041821 Jun 10 13:12:35.092872 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 482/932] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:35 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:35.570418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3dc00103-b3f3-4276-b51b-257cddd6bb4c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:35.571017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3dc00103-b3f3-4276-b51b-257cddd6bb4c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:35.577439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3dc00103-b3f3-4276-b51b-257cddd6bb4c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:35.577879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3dc00103-b3f3-4276-b51b-257cddd6bb4c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:35.578648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3dc00103-b3f3-4276-b51b-257cddd6bb4c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:35.579072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3dc00103-b3f3-4276-b51b-257cddd6bb4c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:35.708932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3dc00103-b3f3-4276-b51b-257cddd6bb4c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:35.716237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3dc00103-b3f3-4276-b51b-257cddd6bb4c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149637 Jun 10 13:12:35.716816 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 451/933] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:35 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:36.103597 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-393d3a00-cd39-4450-93cf-349e7cd47407 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:36.104539 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-393d3a00-cd39-4450-93cf-349e7cd47407 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:36.111008 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-393d3a00-cd39-4450-93cf-349e7cd47407 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:36.111457 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-393d3a00-cd39-4450-93cf-349e7cd47407 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:36.112212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-393d3a00-cd39-4450-93cf-349e7cd47407 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:36.112939 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-393d3a00-cd39-4450-93cf-349e7cd47407 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:36.141165 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-393d3a00-cd39-4450-93cf-349e7cd47407 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041504 Jun 10 13:12:36.141702 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 483/934] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:36 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:36.733795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-616b6ba4-f264-438f-b8a5-dda1cd8726e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:36.734401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-616b6ba4-f264-438f-b8a5-dda1cd8726e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:36.741759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-616b6ba4-f264-438f-b8a5-dda1cd8726e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:36.742205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-616b6ba4-f264-438f-b8a5-dda1cd8726e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:36.742951 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-616b6ba4-f264-438f-b8a5-dda1cd8726e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:36.743376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-616b6ba4-f264-438f-b8a5-dda1cd8726e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:36.870222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-616b6ba4-f264-438f-b8a5-dda1cd8726e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:36.877795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-616b6ba4-f264-438f-b8a5-dda1cd8726e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147880 Jun 10 13:12:36.878356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 452/935] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:36 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:37.153362 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-93d31d69-885c-45ec-89bf-a8d0b980f088 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:37.154226 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-93d31d69-885c-45ec-89bf-a8d0b980f088 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:37.160814 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-93d31d69-885c-45ec-89bf-a8d0b980f088 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:37.161268 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-93d31d69-885c-45ec-89bf-a8d0b980f088 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:37.162612 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-93d31d69-885c-45ec-89bf-a8d0b980f088 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:37.163043 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-93d31d69-885c-45ec-89bf-a8d0b980f088 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:37.191492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-93d31d69-885c-45ec-89bf-a8d0b980f088 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042332 Jun 10 13:12:37.192041 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 484/936] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:37 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:37.894736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-63485e70-745b-46ca-9ac1-00defb5537f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:37.895383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-63485e70-745b-46ca-9ac1-00defb5537f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:37.901973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63485e70-745b-46ca-9ac1-00defb5537f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:37.902419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63485e70-745b-46ca-9ac1-00defb5537f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:37.903174 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63485e70-745b-46ca-9ac1-00defb5537f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:37.903613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63485e70-745b-46ca-9ac1-00defb5537f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:38.033167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-63485e70-745b-46ca-9ac1-00defb5537f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:38.040675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-63485e70-745b-46ca-9ac1-00defb5537f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149786 Jun 10 13:12:38.041283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 453/937] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:37 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:38.205455 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-155324b5-cbaa-423f-ab7f-81b20a157f2b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:38.206387 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-155324b5-cbaa-423f-ab7f-81b20a157f2b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:38.212890 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-155324b5-cbaa-423f-ab7f-81b20a157f2b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:38.213342 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-155324b5-cbaa-423f-ab7f-81b20a157f2b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:38.214110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-155324b5-cbaa-423f-ab7f-81b20a157f2b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:38.214548 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-155324b5-cbaa-423f-ab7f-81b20a157f2b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:38.243943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-155324b5-cbaa-423f-ab7f-81b20a157f2b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042547 Jun 10 13:12:38.244492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 485/938] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:38 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:39.057702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-25405da5-4e89-4efe-87ad-a700d852393f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:39.058326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-25405da5-4e89-4efe-87ad-a700d852393f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:39.064712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25405da5-4e89-4efe-87ad-a700d852393f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:39.065190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25405da5-4e89-4efe-87ad-a700d852393f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:39.065979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25405da5-4e89-4efe-87ad-a700d852393f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:39.066417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25405da5-4e89-4efe-87ad-a700d852393f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:39.069490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:39.194584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-25405da5-4e89-4efe-87ad-a700d852393f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:39.202046 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-25405da5-4e89-4efe-87ad-a700d852393f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148154 Jun 10 13:12:39.202601 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 454/939] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:39 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:39.256150 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ad23f300-a9b7-4cf1-9587-a7a2fb470819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:39.257091 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ad23f300-a9b7-4cf1-9587-a7a2fb470819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:39.262996 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ad23f300-a9b7-4cf1-9587-a7a2fb470819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:39.263432 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ad23f300-a9b7-4cf1-9587-a7a2fb470819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:39.264175 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ad23f300-a9b7-4cf1-9587-a7a2fb470819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:39.264601 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ad23f300-a9b7-4cf1-9587-a7a2fb470819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:39.292677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ad23f300-a9b7-4cf1-9587-a7a2fb470819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041128 Jun 10 13:12:39.293243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 486/940] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:39 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:40.219474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3f40fbda-ee78-4f9a-97f3-4d8a319b1f43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:40.220107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3f40fbda-ee78-4f9a-97f3-4d8a319b1f43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:40.226709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f40fbda-ee78-4f9a-97f3-4d8a319b1f43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:40.227090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f40fbda-ee78-4f9a-97f3-4d8a319b1f43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:40.227840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f40fbda-ee78-4f9a-97f3-4d8a319b1f43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:40.228265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f40fbda-ee78-4f9a-97f3-4d8a319b1f43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:40.304233 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3bc5de55-dc6e-4e59-a8bf-daab4ba56059 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:40.305298 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3bc5de55-dc6e-4e59-a8bf-daab4ba56059 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:40.311920 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3bc5de55-dc6e-4e59-a8bf-daab4ba56059 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:40.312439 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3bc5de55-dc6e-4e59-a8bf-daab4ba56059 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:40.313299 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3bc5de55-dc6e-4e59-a8bf-daab4ba56059 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:40.313803 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3bc5de55-dc6e-4e59-a8bf-daab4ba56059 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:40.344586 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3bc5de55-dc6e-4e59-a8bf-daab4ba56059 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044794 Jun 10 13:12:40.345178 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 487/941] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:40 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:12:40.359315 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f40fbda-ee78-4f9a-97f3-4d8a319b1f43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:40.367724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3f40fbda-ee78-4f9a-97f3-4d8a319b1f43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152038 Jun 10 13:12:40.368284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 455/942] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:40 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:41.355757 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-39b38675-65d1-416a-a64b-885a1a085458 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:41.356671 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-39b38675-65d1-416a-a64b-885a1a085458 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:41.364946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-39b38675-65d1-416a-a64b-885a1a085458 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:41.365513 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-39b38675-65d1-416a-a64b-885a1a085458 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:41.366383 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-39b38675-65d1-416a-a64b-885a1a085458 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:41.366853 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-39b38675-65d1-416a-a64b-885a1a085458 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:41.385202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0f8c7733-ee27-4ee1-a73b-f63bb4d85402 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:41.385799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0f8c7733-ee27-4ee1-a73b-f63bb4d85402 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:41.392309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f8c7733-ee27-4ee1-a73b-f63bb4d85402 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:41.392800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f8c7733-ee27-4ee1-a73b-f63bb4d85402 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:41.393585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f8c7733-ee27-4ee1-a73b-f63bb4d85402 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:41.394013 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f8c7733-ee27-4ee1-a73b-f63bb4d85402 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:41.399081 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-39b38675-65d1-416a-a64b-885a1a085458 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047355 Jun 10 13:12:41.399643 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 488/943] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:41 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:41.532414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0f8c7733-ee27-4ee1-a73b-f63bb4d85402 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:41.539977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0f8c7733-ee27-4ee1-a73b-f63bb4d85402 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.158565 Jun 10 13:12:41.540540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 456/944] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:41 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:12:42.411150 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cbd113cc-de54-4012-9316-1b0bddc340a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:42.412104 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cbd113cc-de54-4012-9316-1b0bddc340a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:42.418502 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbd113cc-de54-4012-9316-1b0bddc340a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:42.418944 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbd113cc-de54-4012-9316-1b0bddc340a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:42.419692 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbd113cc-de54-4012-9316-1b0bddc340a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:42.420117 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbd113cc-de54-4012-9316-1b0bddc340a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:42.449007 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cbd113cc-de54-4012-9316-1b0bddc340a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041736 Jun 10 13:12:42.449544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 489/945] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:42 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:42.557442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-98ca84ee-b7c1-4a41-9b6a-c32fb0bbceed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:42.558045 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-98ca84ee-b7c1-4a41-9b6a-c32fb0bbceed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:42.564469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98ca84ee-b7c1-4a41-9b6a-c32fb0bbceed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:42.564953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98ca84ee-b7c1-4a41-9b6a-c32fb0bbceed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:42.565740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98ca84ee-b7c1-4a41-9b6a-c32fb0bbceed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:42.566165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98ca84ee-b7c1-4a41-9b6a-c32fb0bbceed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:42.698920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-98ca84ee-b7c1-4a41-9b6a-c32fb0bbceed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:42.706412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-98ca84ee-b7c1-4a41-9b6a-c32fb0bbceed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152778 Jun 10 13:12:42.707157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 457/946] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:42 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:43.461497 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-34a75831-8978-42f7-b5dd-be18bd10d00a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:43.462569 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-34a75831-8978-42f7-b5dd-be18bd10d00a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:43.469673 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-34a75831-8978-42f7-b5dd-be18bd10d00a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:43.470694 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-34a75831-8978-42f7-b5dd-be18bd10d00a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:43.471073 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-34a75831-8978-42f7-b5dd-be18bd10d00a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:43.471506 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-34a75831-8978-42f7-b5dd-be18bd10d00a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:43.500291 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-34a75831-8978-42f7-b5dd-be18bd10d00a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043328 Jun 10 13:12:43.500844 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 490/947] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:43 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:43.723632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-52de8de5-40e2-4950-90ca-9b8fc5036277 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:43.724271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-52de8de5-40e2-4950-90ca-9b8fc5036277 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:43.730706 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52de8de5-40e2-4950-90ca-9b8fc5036277 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:43.731152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52de8de5-40e2-4950-90ca-9b8fc5036277 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:43.731906 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52de8de5-40e2-4950-90ca-9b8fc5036277 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:43.732338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52de8de5-40e2-4950-90ca-9b8fc5036277 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:43.861058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-52de8de5-40e2-4950-90ca-9b8fc5036277 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:43.869307 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-52de8de5-40e2-4950-90ca-9b8fc5036277 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149573 Jun 10 13:12:43.869874 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 458/948] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:43 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:44.511482 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-de628592-5730-435e-93f4-b827e27e97a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:44.512398 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-de628592-5730-435e-93f4-b827e27e97a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:44.518908 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-de628592-5730-435e-93f4-b827e27e97a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:44.519359 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-de628592-5730-435e-93f4-b827e27e97a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:44.520123 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-de628592-5730-435e-93f4-b827e27e97a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:44.520561 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-de628592-5730-435e-93f4-b827e27e97a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:44.548702 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-de628592-5730-435e-93f4-b827e27e97a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041194 Jun 10 13:12:44.549304 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 491/949] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:44 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:44.886828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6bb3ed80-c512-4c52-9e1b-b6261de721de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:44.887468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6bb3ed80-c512-4c52-9e1b-b6261de721de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:44.894023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bb3ed80-c512-4c52-9e1b-b6261de721de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:44.894471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bb3ed80-c512-4c52-9e1b-b6261de721de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:44.895238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bb3ed80-c512-4c52-9e1b-b6261de721de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:44.895736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bb3ed80-c512-4c52-9e1b-b6261de721de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:45.020817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6bb3ed80-c512-4c52-9e1b-b6261de721de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:45.028253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6bb3ed80-c512-4c52-9e1b-b6261de721de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145257 Jun 10 13:12:45.028831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 459/950] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:44 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:45.560287 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6bb13f8a-f584-44fd-9dab-9a96e9beede3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:45.561244 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6bb13f8a-f584-44fd-9dab-9a96e9beede3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:45.567689 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6bb13f8a-f584-44fd-9dab-9a96e9beede3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:45.568138 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6bb13f8a-f584-44fd-9dab-9a96e9beede3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:45.568923 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6bb13f8a-f584-44fd-9dab-9a96e9beede3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:45.569355 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6bb13f8a-f584-44fd-9dab-9a96e9beede3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:45.597721 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6bb13f8a-f584-44fd-9dab-9a96e9beede3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041367 Jun 10 13:12:45.598197 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 492/951] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:45 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:46.045759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-da3e1af8-03d0-48ea-8fed-ee6d34398212 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:46.046382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-da3e1af8-03d0-48ea-8fed-ee6d34398212 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:46.052934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-da3e1af8-03d0-48ea-8fed-ee6d34398212 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:46.053391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-da3e1af8-03d0-48ea-8fed-ee6d34398212 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:46.054119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-da3e1af8-03d0-48ea-8fed-ee6d34398212 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:46.054544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-da3e1af8-03d0-48ea-8fed-ee6d34398212 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:46.179928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-da3e1af8-03d0-48ea-8fed-ee6d34398212 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:46.187323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-da3e1af8-03d0-48ea-8fed-ee6d34398212 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145409 Jun 10 13:12:46.187880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 460/952] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:46 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:46.609883 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-75df06d8-cb65-4b54-bb08-bb76140db62e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:46.610717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-75df06d8-cb65-4b54-bb08-bb76140db62e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:46.618168 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-75df06d8-cb65-4b54-bb08-bb76140db62e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:46.618614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-75df06d8-cb65-4b54-bb08-bb76140db62e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:46.619382 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-75df06d8-cb65-4b54-bb08-bb76140db62e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:46.619845 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-75df06d8-cb65-4b54-bb08-bb76140db62e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:46.648310 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-75df06d8-cb65-4b54-bb08-bb76140db62e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042368 Jun 10 13:12:46.648871 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 493/953] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:46 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:47.204908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f62ed78b-5bdf-42b5-a089-625004253048 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:47.205513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f62ed78b-5bdf-42b5-a089-625004253048 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:47.212651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f62ed78b-5bdf-42b5-a089-625004253048 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:47.213121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f62ed78b-5bdf-42b5-a089-625004253048 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:47.213870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f62ed78b-5bdf-42b5-a089-625004253048 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:47.214294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f62ed78b-5bdf-42b5-a089-625004253048 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:47.352998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f62ed78b-5bdf-42b5-a089-625004253048 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:47.361844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f62ed78b-5bdf-42b5-a089-625004253048 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.160895 Jun 10 13:12:47.362699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 461/954] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:47 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:47.660145 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7886eb71-1eb2-4373-a963-31c99ef23ff2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:47.660971 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7886eb71-1eb2-4373-a963-31c99ef23ff2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:47.668064 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7886eb71-1eb2-4373-a963-31c99ef23ff2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:47.668517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7886eb71-1eb2-4373-a963-31c99ef23ff2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:47.669316 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7886eb71-1eb2-4373-a963-31c99ef23ff2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:47.669756 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7886eb71-1eb2-4373-a963-31c99ef23ff2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:47.697420 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7886eb71-1eb2-4373-a963-31c99ef23ff2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041450 Jun 10 13:12:47.697960 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 494/955] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:47 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:48.395427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5340732f-abd8-4925-820b-e22103537658 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:48.396034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5340732f-abd8-4925-820b-e22103537658 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:48.402650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5340732f-abd8-4925-820b-e22103537658 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:48.403092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5340732f-abd8-4925-820b-e22103537658 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:48.403846 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5340732f-abd8-4925-820b-e22103537658 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:48.404275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5340732f-abd8-4925-820b-e22103537658 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:48.529181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5340732f-abd8-4925-820b-e22103537658 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:48.536577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5340732f-abd8-4925-820b-e22103537658 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145048 Jun 10 13:12:48.537179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 462/956] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:48 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:48.709314 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-04f47f27-e1b7-43f0-a947-0b398f6bf765 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:48.710228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-04f47f27-e1b7-43f0-a947-0b398f6bf765 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:48.716638 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04f47f27-e1b7-43f0-a947-0b398f6bf765 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:48.717118 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04f47f27-e1b7-43f0-a947-0b398f6bf765 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:48.721627 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04f47f27-e1b7-43f0-a947-0b398f6bf765 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:48.721627 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04f47f27-e1b7-43f0-a947-0b398f6bf765 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:48.721860 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:48.748101 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-04f47f27-e1b7-43f0-a947-0b398f6bf765 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042745 Jun 10 13:12:48.756005 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 495/957] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:48 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:49.553554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d99111b0-b077-4f2c-8e74-d42a4dbe0fed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:49.554155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d99111b0-b077-4f2c-8e74-d42a4dbe0fed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:49.556470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=52 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:49.561236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d99111b0-b077-4f2c-8e74-d42a4dbe0fed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:49.561681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d99111b0-b077-4f2c-8e74-d42a4dbe0fed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:49.562435 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d99111b0-b077-4f2c-8e74-d42a4dbe0fed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:49.562867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d99111b0-b077-4f2c-8e74-d42a4dbe0fed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:49.690739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d99111b0-b077-4f2c-8e74-d42a4dbe0fed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:49.698898 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d99111b0-b077-4f2c-8e74-d42a4dbe0fed tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149216 Jun 10 13:12:49.699778 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 463/958] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:49 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:49.760075 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-62f56522-0691-49b4-bfd1-768bdc4574d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:49.761014 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-62f56522-0691-49b4-bfd1-768bdc4574d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:49.768099 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62f56522-0691-49b4-bfd1-768bdc4574d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:49.768534 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62f56522-0691-49b4-bfd1-768bdc4574d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:49.769333 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62f56522-0691-49b4-bfd1-768bdc4574d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:49.769769 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62f56522-0691-49b4-bfd1-768bdc4574d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:49.798077 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-62f56522-0691-49b4-bfd1-768bdc4574d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042029 Jun 10 13:12:49.798618 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 496/959] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:49 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:50.716232 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-63f91a81-90d9-4f26-b238-3ddefbcc3bea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:50.716958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-63f91a81-90d9-4f26-b238-3ddefbcc3bea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:50.724530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63f91a81-90d9-4f26-b238-3ddefbcc3bea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:50.725011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63f91a81-90d9-4f26-b238-3ddefbcc3bea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:50.725767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63f91a81-90d9-4f26-b238-3ddefbcc3bea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:50.726197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63f91a81-90d9-4f26-b238-3ddefbcc3bea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:50.809475 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-965503d0-b549-4593-ba2a-89ec4bb5bb47 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:50.810385 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-965503d0-b549-4593-ba2a-89ec4bb5bb47 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:50.817024 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-965503d0-b549-4593-ba2a-89ec4bb5bb47 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:50.817541 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-965503d0-b549-4593-ba2a-89ec4bb5bb47 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:50.818303 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-965503d0-b549-4593-ba2a-89ec4bb5bb47 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:50.818736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-965503d0-b549-4593-ba2a-89ec4bb5bb47 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:50.849463 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-965503d0-b549-4593-ba2a-89ec4bb5bb47 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043176 Jun 10 13:12:50.850220 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 497/960] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:50 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:12:50.853739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-63f91a81-90d9-4f26-b238-3ddefbcc3bea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:50.861263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-63f91a81-90d9-4f26-b238-3ddefbcc3bea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148833 Jun 10 13:12:50.861811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 464/961] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:50 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:51.862068 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-02f9d8f3-9fcb-49db-8742-94ef53aff9b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:51.868047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-02f9d8f3-9fcb-49db-8742-94ef53aff9b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:51.869831 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-02f9d8f3-9fcb-49db-8742-94ef53aff9b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:51.870270 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-02f9d8f3-9fcb-49db-8742-94ef53aff9b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:51.871019 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-02f9d8f3-9fcb-49db-8742-94ef53aff9b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:51.871556 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-02f9d8f3-9fcb-49db-8742-94ef53aff9b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:51.879176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f6e1d451-e03f-49b0-ad5e-05931be6fa78 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:51.880429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f6e1d451-e03f-49b0-ad5e-05931be6fa78 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:51.887014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f6e1d451-e03f-49b0-ad5e-05931be6fa78 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:51.887460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f6e1d451-e03f-49b0-ad5e-05931be6fa78 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:51.888220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f6e1d451-e03f-49b0-ad5e-05931be6fa78 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:51.888653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f6e1d451-e03f-49b0-ad5e-05931be6fa78 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:51.901465 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-02f9d8f3-9fcb-49db-8742-94ef53aff9b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044244 Jun 10 13:12:51.902008 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 498/962] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:51 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:52.017484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f6e1d451-e03f-49b0-ad5e-05931be6fa78 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:52.025561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f6e1d451-e03f-49b0-ad5e-05931be6fa78 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150357 Jun 10 13:12:52.026122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 465/963] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:51 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:12:52.914005 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-176f1b3c-725c-4b42-92d6-5fcdf72970a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:52.914906 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-176f1b3c-725c-4b42-92d6-5fcdf72970a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:52.921420 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-176f1b3c-725c-4b42-92d6-5fcdf72970a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:52.921863 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-176f1b3c-725c-4b42-92d6-5fcdf72970a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:52.922617 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-176f1b3c-725c-4b42-92d6-5fcdf72970a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:52.923043 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-176f1b3c-725c-4b42-92d6-5fcdf72970a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:52.950897 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-176f1b3c-725c-4b42-92d6-5fcdf72970a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041671 Jun 10 13:12:52.951445 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 499/964] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:52 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:53.042189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0972c748-4959-4da8-b75e-849068d41662 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:53.042813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0972c748-4959-4da8-b75e-849068d41662 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:53.049269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0972c748-4959-4da8-b75e-849068d41662 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:53.049722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0972c748-4959-4da8-b75e-849068d41662 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:53.050489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0972c748-4959-4da8-b75e-849068d41662 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:53.050926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0972c748-4959-4da8-b75e-849068d41662 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:53.177915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0972c748-4959-4da8-b75e-849068d41662 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:53.185453 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0972c748-4959-4da8-b75e-849068d41662 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147235 Jun 10 13:12:53.186004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 466/965] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:53 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:53.963269 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-61f96330-587c-42ab-a62f-310a0b156b4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:53.964175 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-61f96330-587c-42ab-a62f-310a0b156b4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:53.970645 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-61f96330-587c-42ab-a62f-310a0b156b4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:53.971081 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-61f96330-587c-42ab-a62f-310a0b156b4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:53.971830 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-61f96330-587c-42ab-a62f-310a0b156b4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:53.972256 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-61f96330-587c-42ab-a62f-310a0b156b4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:54.000802 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-61f96330-587c-42ab-a62f-310a0b156b4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041514 Jun 10 13:12:54.001331 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 500/966] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:53 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:54.201882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-14fd8bba-8c36-4d3c-81bc-0b0422f3d692 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:54.202926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-14fd8bba-8c36-4d3c-81bc-0b0422f3d692 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:54.211885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-14fd8bba-8c36-4d3c-81bc-0b0422f3d692 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:54.212360 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-14fd8bba-8c36-4d3c-81bc-0b0422f3d692 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:54.213172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-14fd8bba-8c36-4d3c-81bc-0b0422f3d692 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:54.213671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-14fd8bba-8c36-4d3c-81bc-0b0422f3d692 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:54.343056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-14fd8bba-8c36-4d3c-81bc-0b0422f3d692 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:54.350623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-14fd8bba-8c36-4d3c-81bc-0b0422f3d692 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152634 Jun 10 13:12:54.351181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 467/967] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:54 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:55.011607 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ea4a8368-3595-4be6-ae49-2f7378bcafa8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:55.012568 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ea4a8368-3595-4be6-ae49-2f7378bcafa8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:55.018984 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ea4a8368-3595-4be6-ae49-2f7378bcafa8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:55.019424 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ea4a8368-3595-4be6-ae49-2f7378bcafa8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:55.020170 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ea4a8368-3595-4be6-ae49-2f7378bcafa8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:55.020597 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ea4a8368-3595-4be6-ae49-2f7378bcafa8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:55.048540 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ea4a8368-3595-4be6-ae49-2f7378bcafa8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040970 Jun 10 13:12:55.049099 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 501/968] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:55 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:55.367796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cf43761d-df4b-46d8-865f-be335150aa49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:55.368402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cf43761d-df4b-46d8-865f-be335150aa49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:55.374893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf43761d-df4b-46d8-865f-be335150aa49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:55.375337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf43761d-df4b-46d8-865f-be335150aa49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:55.376092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf43761d-df4b-46d8-865f-be335150aa49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:55.376522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf43761d-df4b-46d8-865f-be335150aa49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:55.506654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cf43761d-df4b-46d8-865f-be335150aa49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:55.514367 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cf43761d-df4b-46d8-865f-be335150aa49 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150455 Jun 10 13:12:55.514922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 468/969] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:55 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:56.060784 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1bf4affa-e691-436a-98ad-b41d96b15310 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:56.061759 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1bf4affa-e691-436a-98ad-b41d96b15310 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:56.068840 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1bf4affa-e691-436a-98ad-b41d96b15310 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:56.069309 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1bf4affa-e691-436a-98ad-b41d96b15310 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:56.070088 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1bf4affa-e691-436a-98ad-b41d96b15310 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:56.070519 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1bf4affa-e691-436a-98ad-b41d96b15310 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:56.097859 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1bf4affa-e691-436a-98ad-b41d96b15310 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041140 Jun 10 13:12:56.098389 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 502/970] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:56 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:56.531997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9e6adb7d-57a0-422c-9846-1cc5dfa25b43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:56.532602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9e6adb7d-57a0-422c-9846-1cc5dfa25b43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:56.539556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e6adb7d-57a0-422c-9846-1cc5dfa25b43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:56.540008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e6adb7d-57a0-422c-9846-1cc5dfa25b43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:56.540783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e6adb7d-57a0-422c-9846-1cc5dfa25b43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:56.541219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e6adb7d-57a0-422c-9846-1cc5dfa25b43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:56.667490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9e6adb7d-57a0-422c-9846-1cc5dfa25b43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:56.675011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9e6adb7d-57a0-422c-9846-1cc5dfa25b43 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146922 Jun 10 13:12:56.675565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 469/971] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:56 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:57.109408 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c93104d2-6f63-4ed2-8309-c7fc4d33f4d1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:57.110317 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c93104d2-6f63-4ed2-8309-c7fc4d33f4d1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:57.116981 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c93104d2-6f63-4ed2-8309-c7fc4d33f4d1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:57.117426 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c93104d2-6f63-4ed2-8309-c7fc4d33f4d1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:57.118173 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c93104d2-6f63-4ed2-8309-c7fc4d33f4d1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:57.118600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c93104d2-6f63-4ed2-8309-c7fc4d33f4d1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:57.147607 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c93104d2-6f63-4ed2-8309-c7fc4d33f4d1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042135 Jun 10 13:12:57.148515 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 503/972] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:57 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:57.691333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-891cce60-30a3-4885-8832-ce8146067d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:57.691936 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-891cce60-30a3-4885-8832-ce8146067d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:57.699019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-891cce60-30a3-4885-8832-ce8146067d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:57.699465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-891cce60-30a3-4885-8832-ce8146067d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:57.700221 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-891cce60-30a3-4885-8832-ce8146067d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:57.700647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-891cce60-30a3-4885-8832-ce8146067d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:57.827433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-891cce60-30a3-4885-8832-ce8146067d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:57.829676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:57.835514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-891cce60-30a3-4885-8832-ce8146067d47 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148010 Jun 10 13:12:57.836083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 470/973] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:57 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:58.158856 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0bb090ff-21cb-4560-adc3-b1ee7431e7de tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:58.159761 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0bb090ff-21cb-4560-adc3-b1ee7431e7de tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:58.166123 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0bb090ff-21cb-4560-adc3-b1ee7431e7de tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:58.166562 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0bb090ff-21cb-4560-adc3-b1ee7431e7de tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:58.167309 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0bb090ff-21cb-4560-adc3-b1ee7431e7de tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:58.167735 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0bb090ff-21cb-4560-adc3-b1ee7431e7de tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:58.196668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0bb090ff-21cb-4560-adc3-b1ee7431e7de tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041811 Jun 10 13:12:58.197284 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 504/974] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:58 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:12:58.853089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c0d47968-1afb-4246-aee6-b19595debf35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:58.853696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c0d47968-1afb-4246-aee6-b19595debf35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:58.861053 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c0d47968-1afb-4246-aee6-b19595debf35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:58.861504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c0d47968-1afb-4246-aee6-b19595debf35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:58.862554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c0d47968-1afb-4246-aee6-b19595debf35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:58.862994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c0d47968-1afb-4246-aee6-b19595debf35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:58.989713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c0d47968-1afb-4246-aee6-b19595debf35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:12:58.997104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c0d47968-1afb-4246-aee6-b19595debf35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148455 Jun 10 13:12:58.997673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 471/975] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:12:58 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:12:59.208268 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ac7130dd-ab18-4ae5-baee-6da7c9641e4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:12:59.209330 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ac7130dd-ab18-4ae5-baee-6da7c9641e4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:12:59.211742 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=60 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:59.216443 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ac7130dd-ab18-4ae5-baee-6da7c9641e4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:59.216976 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ac7130dd-ab18-4ae5-baee-6da7c9641e4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:59.217737 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ac7130dd-ab18-4ae5-baee-6da7c9641e4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:12:59.218168 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ac7130dd-ab18-4ae5-baee-6da7c9641e4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:12:59.247513 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ac7130dd-ab18-4ae5-baee-6da7c9641e4e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043734 Jun 10 13:12:59.248061 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 505/976] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:12:59 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:00.014168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ebe486a1-116a-4989-90eb-8f87d3cc156e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:00.014791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ebe486a1-116a-4989-90eb-8f87d3cc156e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:00.021277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebe486a1-116a-4989-90eb-8f87d3cc156e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:00.021755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebe486a1-116a-4989-90eb-8f87d3cc156e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:00.022513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebe486a1-116a-4989-90eb-8f87d3cc156e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:00.022943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebe486a1-116a-4989-90eb-8f87d3cc156e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:00.147076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ebe486a1-116a-4989-90eb-8f87d3cc156e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:00.154734 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ebe486a1-116a-4989-90eb-8f87d3cc156e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.144358 Jun 10 13:13:00.155327 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 472/977] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:00 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:00.259230 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a211fbed-5a14-49e5-bfc9-23610e58ee20 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:00.260117 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a211fbed-5a14-49e5-bfc9-23610e58ee20 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:00.266823 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a211fbed-5a14-49e5-bfc9-23610e58ee20 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:00.267271 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a211fbed-5a14-49e5-bfc9-23610e58ee20 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:00.268031 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a211fbed-5a14-49e5-bfc9-23610e58ee20 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:00.268466 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a211fbed-5a14-49e5-bfc9-23610e58ee20 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:00.300356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a211fbed-5a14-49e5-bfc9-23610e58ee20 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045124 Jun 10 13:13:00.300924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 506/978] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:00 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:01.172685 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fab2f066-dbbf-442c-8513-36e458802429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:01.173322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fab2f066-dbbf-442c-8513-36e458802429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:01.179909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fab2f066-dbbf-442c-8513-36e458802429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:01.180356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fab2f066-dbbf-442c-8513-36e458802429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:01.181224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fab2f066-dbbf-442c-8513-36e458802429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:01.181653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fab2f066-dbbf-442c-8513-36e458802429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:01.309549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fab2f066-dbbf-442c-8513-36e458802429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:01.312790 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b5dc05f0-d5b3-4620-a936-c325c3f16679 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:01.313818 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b5dc05f0-d5b3-4620-a936-c325c3f16679 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:01.317066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fab2f066-dbbf-442c-8513-36e458802429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148384 Jun 10 13:13:01.318191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 473/979] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:01 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:01.321066 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b5dc05f0-d5b3-4620-a936-c325c3f16679 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:01.321519 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b5dc05f0-d5b3-4620-a936-c325c3f16679 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:01.322302 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b5dc05f0-d5b3-4620-a936-c325c3f16679 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:01.322744 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b5dc05f0-d5b3-4620-a936-c325c3f16679 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:01.351927 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b5dc05f0-d5b3-4620-a936-c325c3f16679 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043140 Jun 10 13:13:01.352481 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 507/980] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:01 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:13:02.335204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-772211ac-1af6-4e1e-8eee-bd4351b5dc45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:02.335817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-772211ac-1af6-4e1e-8eee-bd4351b5dc45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:02.342330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-772211ac-1af6-4e1e-8eee-bd4351b5dc45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:02.342782 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-772211ac-1af6-4e1e-8eee-bd4351b5dc45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:02.343551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-772211ac-1af6-4e1e-8eee-bd4351b5dc45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:02.343994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-772211ac-1af6-4e1e-8eee-bd4351b5dc45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:02.363842 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b388c557-5f94-4769-8d8b-2dbff7787d31 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:02.364754 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b388c557-5f94-4769-8d8b-2dbff7787d31 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:02.371270 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b388c557-5f94-4769-8d8b-2dbff7787d31 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:02.371717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b388c557-5f94-4769-8d8b-2dbff7787d31 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:02.372477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b388c557-5f94-4769-8d8b-2dbff7787d31 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:02.372937 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b388c557-5f94-4769-8d8b-2dbff7787d31 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:02.402613 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b388c557-5f94-4769-8d8b-2dbff7787d31 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042932 Jun 10 13:13:02.403166 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 508/981] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:02 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:13:02.473836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-772211ac-1af6-4e1e-8eee-bd4351b5dc45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:02.481378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-772211ac-1af6-4e1e-8eee-bd4351b5dc45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150597 Jun 10 13:13:02.481938 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 474/982] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:02 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:03.414360 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2c4aa1b6-c280-4cdf-b091-7c3db1bd3ac2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:03.415387 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2c4aa1b6-c280-4cdf-b091-7c3db1bd3ac2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:03.422149 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c4aa1b6-c280-4cdf-b091-7c3db1bd3ac2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:03.422598 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c4aa1b6-c280-4cdf-b091-7c3db1bd3ac2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:03.423362 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c4aa1b6-c280-4cdf-b091-7c3db1bd3ac2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:03.423795 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c4aa1b6-c280-4cdf-b091-7c3db1bd3ac2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:03.453143 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2c4aa1b6-c280-4cdf-b091-7c3db1bd3ac2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042747 Jun 10 13:13:03.453686 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 509/983] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:03 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:03.498053 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6c850f8d-db5e-4775-959d-a4d568c6d054 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:03.498658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6c850f8d-db5e-4775-959d-a4d568c6d054 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:03.505834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c850f8d-db5e-4775-959d-a4d568c6d054 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:03.506288 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c850f8d-db5e-4775-959d-a4d568c6d054 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:03.507042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c850f8d-db5e-4775-959d-a4d568c6d054 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:03.507471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c850f8d-db5e-4775-959d-a4d568c6d054 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:03.638884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6c850f8d-db5e-4775-959d-a4d568c6d054 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:03.646611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6c850f8d-db5e-4775-959d-a4d568c6d054 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152334 Jun 10 13:13:03.647171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 475/984] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:03 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:04.335989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:13:04.336204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:13:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:13:04.465136 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3449a1b1-ba40-4f7b-902d-45de620312e3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:04.466056 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3449a1b1-ba40-4f7b-902d-45de620312e3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:04.472504 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3449a1b1-ba40-4f7b-902d-45de620312e3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:04.472993 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3449a1b1-ba40-4f7b-902d-45de620312e3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:04.474404 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3449a1b1-ba40-4f7b-902d-45de620312e3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:04.474838 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3449a1b1-ba40-4f7b-902d-45de620312e3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:04.502765 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3449a1b1-ba40-4f7b-902d-45de620312e3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041648 Jun 10 13:13:04.503299 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 510/985] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:04 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:04.663455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7f6fa988-8cc3-4149-8cf7-881987ca4647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:04.663997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7f6fa988-8cc3-4149-8cf7-881987ca4647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:04.670570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f6fa988-8cc3-4149-8cf7-881987ca4647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:04.671018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f6fa988-8cc3-4149-8cf7-881987ca4647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:04.671775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f6fa988-8cc3-4149-8cf7-881987ca4647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:04.672207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f6fa988-8cc3-4149-8cf7-881987ca4647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:04.796709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7f6fa988-8cc3-4149-8cf7-881987ca4647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:04.804156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7f6fa988-8cc3-4149-8cf7-881987ca4647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.144639 Jun 10 13:13:04.804710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 476/986] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:04 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:05.514887 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-049d1705-da79-4108-b361-8d607d75f231 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:05.515805 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-049d1705-da79-4108-b361-8d607d75f231 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:05.522268 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-049d1705-da79-4108-b361-8d607d75f231 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:05.522712 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-049d1705-da79-4108-b361-8d607d75f231 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:05.523474 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-049d1705-da79-4108-b361-8d607d75f231 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:05.523904 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-049d1705-da79-4108-b361-8d607d75f231 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:05.548690 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:05.553721 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-049d1705-da79-4108-b361-8d607d75f231 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042822 Jun 10 13:13:05.554265 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 511/987] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:05 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:05.821043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c5f73415-e97a-4485-978e-cae8c94d3a2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:05.821699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c5f73415-e97a-4485-978e-cae8c94d3a2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:05.828231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5f73415-e97a-4485-978e-cae8c94d3a2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:05.828672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5f73415-e97a-4485-978e-cae8c94d3a2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:05.829450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5f73415-e97a-4485-978e-cae8c94d3a2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:05.829876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5f73415-e97a-4485-978e-cae8c94d3a2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:05.956398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c5f73415-e97a-4485-978e-cae8c94d3a2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:05.964334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c5f73415-e97a-4485-978e-cae8c94d3a2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147288 Jun 10 13:13:05.964909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 477/988] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:05 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:06.565358 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-eaec158a-046f-42b3-9e05-02e9204e9ce3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:06.566265 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-eaec158a-046f-42b3-9e05-02e9204e9ce3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:06.573187 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eaec158a-046f-42b3-9e05-02e9204e9ce3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:06.573625 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eaec158a-046f-42b3-9e05-02e9204e9ce3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:06.574377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eaec158a-046f-42b3-9e05-02e9204e9ce3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:06.574806 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eaec158a-046f-42b3-9e05-02e9204e9ce3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:06.606474 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-eaec158a-046f-42b3-9e05-02e9204e9ce3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044948 Jun 10 13:13:06.606944 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 512/989] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:06 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:06.981747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4a246fd0-3a81-4a64-b8ce-fc3bfb42e65e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:06.982361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4a246fd0-3a81-4a64-b8ce-fc3bfb42e65e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:06.989192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a246fd0-3a81-4a64-b8ce-fc3bfb42e65e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:06.989637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a246fd0-3a81-4a64-b8ce-fc3bfb42e65e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:06.990395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a246fd0-3a81-4a64-b8ce-fc3bfb42e65e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:06.990830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a246fd0-3a81-4a64-b8ce-fc3bfb42e65e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:07.115288 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4a246fd0-3a81-4a64-b8ce-fc3bfb42e65e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:07.122773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4a246fd0-3a81-4a64-b8ce-fc3bfb42e65e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.144933 Jun 10 13:13:07.123361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 478/990] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:06 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:07.622692 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-bd8cfaa8-55d9-4818-a437-936ed3d3c074 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:07.624096 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-bd8cfaa8-55d9-4818-a437-936ed3d3c074 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:07.636788 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bd8cfaa8-55d9-4818-a437-936ed3d3c074 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:07.637246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bd8cfaa8-55d9-4818-a437-936ed3d3c074 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:07.638655 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bd8cfaa8-55d9-4818-a437-936ed3d3c074 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:07.639100 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bd8cfaa8-55d9-4818-a437-936ed3d3c074 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:07.669422 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-bd8cfaa8-55d9-4818-a437-936ed3d3c074 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051652 Jun 10 13:13:07.669975 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 513/991] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:07 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:08.140475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4b253d61-8a63-44e2-8975-33dee772b267 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:08.141101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4b253d61-8a63-44e2-8975-33dee772b267 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:08.147485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b253d61-8a63-44e2-8975-33dee772b267 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:08.147923 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b253d61-8a63-44e2-8975-33dee772b267 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:08.148669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b253d61-8a63-44e2-8975-33dee772b267 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:08.149128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b253d61-8a63-44e2-8975-33dee772b267 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:08.283927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4b253d61-8a63-44e2-8975-33dee772b267 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:08.290511 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4b253d61-8a63-44e2-8975-33dee772b267 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.153926 Jun 10 13:13:08.291092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 479/992] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:08 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:08.680747 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6f920f6c-951f-4b41-ba7e-9713bb304558 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:08.681627 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6f920f6c-951f-4b41-ba7e-9713bb304558 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:08.688129 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6f920f6c-951f-4b41-ba7e-9713bb304558 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:08.688610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6f920f6c-951f-4b41-ba7e-9713bb304558 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:08.689395 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6f920f6c-951f-4b41-ba7e-9713bb304558 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:08.689829 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6f920f6c-951f-4b41-ba7e-9713bb304558 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:08.718728 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6f920f6c-951f-4b41-ba7e-9713bb304558 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041985 Jun 10 13:13:08.719377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 514/993] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:08 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:09.307349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d4e45048-80cc-47c7-ab18-3a6ab396b98f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:09.307935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d4e45048-80cc-47c7-ab18-3a6ab396b98f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:09.314549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4e45048-80cc-47c7-ab18-3a6ab396b98f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:09.314993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4e45048-80cc-47c7-ab18-3a6ab396b98f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:09.315743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4e45048-80cc-47c7-ab18-3a6ab396b98f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:09.316166 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4e45048-80cc-47c7-ab18-3a6ab396b98f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:09.445779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d4e45048-80cc-47c7-ab18-3a6ab396b98f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:09.453189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d4e45048-80cc-47c7-ab18-3a6ab396b98f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149611 Jun 10 13:13:09.453736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 480/994] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:09 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:09.730725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-412128fb-8e80-4b56-a3c1-2980dd13b806 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:09.731647 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-412128fb-8e80-4b56-a3c1-2980dd13b806 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:09.738164 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-412128fb-8e80-4b56-a3c1-2980dd13b806 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:09.738614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-412128fb-8e80-4b56-a3c1-2980dd13b806 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:09.740021 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-412128fb-8e80-4b56-a3c1-2980dd13b806 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:09.740458 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-412128fb-8e80-4b56-a3c1-2980dd13b806 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:09.768668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-412128fb-8e80-4b56-a3c1-2980dd13b806 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041973 Jun 10 13:13:09.769370 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 515/995] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:09 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:10.470147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5a8ff76d-b2f2-4a6d-b8c4-6c1a7650bbd3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:10.470746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5a8ff76d-b2f2-4a6d-b8c4-6c1a7650bbd3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:10.477815 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5a8ff76d-b2f2-4a6d-b8c4-6c1a7650bbd3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:10.478256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5a8ff76d-b2f2-4a6d-b8c4-6c1a7650bbd3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:10.479014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5a8ff76d-b2f2-4a6d-b8c4-6c1a7650bbd3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:10.479440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5a8ff76d-b2f2-4a6d-b8c4-6c1a7650bbd3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:10.631959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5a8ff76d-b2f2-4a6d-b8c4-6c1a7650bbd3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:10.631959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5a8ff76d-b2f2-4a6d-b8c4-6c1a7650bbd3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.165072 Jun 10 13:13:10.631959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 481/996] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:10 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:10.780685 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b98ce3f0-18bf-44b7-af66-aaa226931edd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:10.781638 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b98ce3f0-18bf-44b7-af66-aaa226931edd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:10.787999 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b98ce3f0-18bf-44b7-af66-aaa226931edd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:10.788439 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b98ce3f0-18bf-44b7-af66-aaa226931edd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:10.789231 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b98ce3f0-18bf-44b7-af66-aaa226931edd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:10.789661 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b98ce3f0-18bf-44b7-af66-aaa226931edd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:10.818145 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b98ce3f0-18bf-44b7-af66-aaa226931edd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041486 Jun 10 13:13:10.818683 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 516/997] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:10 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:11.648254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-36f069d6-5f6e-40e3-adbb-363268653281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:11.648790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-36f069d6-5f6e-40e3-adbb-363268653281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:11.655364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36f069d6-5f6e-40e3-adbb-363268653281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:11.655827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36f069d6-5f6e-40e3-adbb-363268653281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:11.656583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36f069d6-5f6e-40e3-adbb-363268653281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:11.657051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36f069d6-5f6e-40e3-adbb-363268653281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:11.784674 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-36f069d6-5f6e-40e3-adbb-363268653281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:11.792172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-36f069d6-5f6e-40e3-adbb-363268653281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147875 Jun 10 13:13:11.792991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 482/998] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:11 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:11.829626 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-fbe0327e-8ed8-483b-be4c-de23846f716f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:11.830559 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-fbe0327e-8ed8-483b-be4c-de23846f716f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:11.837003 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fbe0327e-8ed8-483b-be4c-de23846f716f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:11.837446 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fbe0327e-8ed8-483b-be4c-de23846f716f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:11.838194 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fbe0327e-8ed8-483b-be4c-de23846f716f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:11.838617 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fbe0327e-8ed8-483b-be4c-de23846f716f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:11.867133 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-fbe0327e-8ed8-483b-be4c-de23846f716f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041434 Jun 10 13:13:11.867671 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 517/999] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:11 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:12.809521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5f9b3c5a-79a3-4dfc-8e3d-f3b4f0da6c09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:12.810123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5f9b3c5a-79a3-4dfc-8e3d-f3b4f0da6c09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:12.816520 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5f9b3c5a-79a3-4dfc-8e3d-f3b4f0da6c09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:12.817509 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5f9b3c5a-79a3-4dfc-8e3d-f3b4f0da6c09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:12.818257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5f9b3c5a-79a3-4dfc-8e3d-f3b4f0da6c09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:12.818681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5f9b3c5a-79a3-4dfc-8e3d-f3b4f0da6c09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:12.878385 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-06a60cd9-0b1a-490f-acd5-6fd28c0133cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:12.879797 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-06a60cd9-0b1a-490f-acd5-6fd28c0133cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:12.886526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-06a60cd9-0b1a-490f-acd5-6fd28c0133cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:12.886977 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-06a60cd9-0b1a-490f-acd5-6fd28c0133cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:12.887749 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-06a60cd9-0b1a-490f-acd5-6fd28c0133cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:12.888187 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-06a60cd9-0b1a-490f-acd5-6fd28c0133cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:12.918610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-06a60cd9-0b1a-490f-acd5-6fd28c0133cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044133 Jun 10 13:13:12.919172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 518/1000] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:12 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:13:12.945985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5f9b3c5a-79a3-4dfc-8e3d-f3b4f0da6c09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:12.953454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5f9b3c5a-79a3-4dfc-8e3d-f3b4f0da6c09 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147728 Jun 10 13:13:12.953999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 483/1001] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:12 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:13.931057 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0d1f4993-150f-477f-8fb9-e95f3d2b166f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:13.932052 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0d1f4993-150f-477f-8fb9-e95f3d2b166f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:13.939330 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0d1f4993-150f-477f-8fb9-e95f3d2b166f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:13.943969 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0d1f4993-150f-477f-8fb9-e95f3d2b166f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:13.943969 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0d1f4993-150f-477f-8fb9-e95f3d2b166f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:13.943969 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0d1f4993-150f-477f-8fb9-e95f3d2b166f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:13.971063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6aa53173-2b24-4080-8ecf-f143a77a7ba7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:13.975542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6aa53173-2b24-4080-8ecf-f143a77a7ba7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:13.976267 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0d1f4993-150f-477f-8fb9-e95f3d2b166f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049874 Jun 10 13:13:13.976849 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 519/1002] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:13 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:13:13.978225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6aa53173-2b24-4080-8ecf-f143a77a7ba7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:13.978776 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6aa53173-2b24-4080-8ecf-f143a77a7ba7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:13.979540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6aa53173-2b24-4080-8ecf-f143a77a7ba7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:13.979969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6aa53173-2b24-4080-8ecf-f143a77a7ba7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:14.105558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6aa53173-2b24-4080-8ecf-f143a77a7ba7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:14.113202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6aa53173-2b24-4080-8ecf-f143a77a7ba7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146039 Jun 10 13:13:14.114360 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 484/1003] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:13 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:14.987051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f926cce4-c13d-430d-8df8-0eab98542f94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:14.988206 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f926cce4-c13d-430d-8df8-0eab98542f94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:14.995285 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f926cce4-c13d-430d-8df8-0eab98542f94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:14.995733 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f926cce4-c13d-430d-8df8-0eab98542f94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:14.996492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f926cce4-c13d-430d-8df8-0eab98542f94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:14.996958 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f926cce4-c13d-430d-8df8-0eab98542f94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:15.025607 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f926cce4-c13d-430d-8df8-0eab98542f94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042607 Jun 10 13:13:15.026136 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 520/1004] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:14 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:15.131246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b86ea931-da60-4a38-ab90-de19866a1641 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:15.131906 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b86ea931-da60-4a38-ab90-de19866a1641 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:15.138287 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b86ea931-da60-4a38-ab90-de19866a1641 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:15.138727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b86ea931-da60-4a38-ab90-de19866a1641 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:15.139528 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b86ea931-da60-4a38-ab90-de19866a1641 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:15.139957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b86ea931-da60-4a38-ab90-de19866a1641 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:15.267408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b86ea931-da60-4a38-ab90-de19866a1641 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:15.275394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b86ea931-da60-4a38-ab90-de19866a1641 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148487 Jun 10 13:13:15.275947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 485/1005] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:15 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:16.037111 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-df0ec2e5-4c05-4a66-b801-ae769e68730c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:16.038024 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-df0ec2e5-4c05-4a66-b801-ae769e68730c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:16.044369 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-df0ec2e5-4c05-4a66-b801-ae769e68730c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:16.044835 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-df0ec2e5-4c05-4a66-b801-ae769e68730c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:16.045589 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-df0ec2e5-4c05-4a66-b801-ae769e68730c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:16.046017 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-df0ec2e5-4c05-4a66-b801-ae769e68730c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:16.073994 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-df0ec2e5-4c05-4a66-b801-ae769e68730c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040930 Jun 10 13:13:16.074530 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 521/1006] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:16 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:16.292239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e0b39376-e1f3-4d12-b783-c3350135c143 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:16.292919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e0b39376-e1f3-4d12-b783-c3350135c143 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:16.299181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0b39376-e1f3-4d12-b783-c3350135c143 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:16.299624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0b39376-e1f3-4d12-b783-c3350135c143 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:16.300371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0b39376-e1f3-4d12-b783-c3350135c143 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:16.300818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0b39376-e1f3-4d12-b783-c3350135c143 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:16.431027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e0b39376-e1f3-4d12-b783-c3350135c143 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:16.438418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e0b39376-e1f3-4d12-b783-c3350135c143 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150110 Jun 10 13:13:16.438978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 486/1007] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:16 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:17.085868 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-533a320f-c352-43ca-8aa1-ab60bb0f944c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:17.086699 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-533a320f-c352-43ca-8aa1-ab60bb0f944c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:17.093141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-533a320f-c352-43ca-8aa1-ab60bb0f944c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:17.093584 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-533a320f-c352-43ca-8aa1-ab60bb0f944c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:17.094341 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-533a320f-c352-43ca-8aa1-ab60bb0f944c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:17.094772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-533a320f-c352-43ca-8aa1-ab60bb0f944c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:17.123409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-533a320f-c352-43ca-8aa1-ab60bb0f944c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041684 Jun 10 13:13:17.123940 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 522/1008] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:17 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:17.455352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-db61506c-599b-41ac-9c22-aab24ea6a284 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:17.455989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-db61506c-599b-41ac-9c22-aab24ea6a284 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:17.463261 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-db61506c-599b-41ac-9c22-aab24ea6a284 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:17.463719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-db61506c-599b-41ac-9c22-aab24ea6a284 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:17.464515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-db61506c-599b-41ac-9c22-aab24ea6a284 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:17.464979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-db61506c-599b-41ac-9c22-aab24ea6a284 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:17.479400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:17.598003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-db61506c-599b-41ac-9c22-aab24ea6a284 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:17.605472 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-db61506c-599b-41ac-9c22-aab24ea6a284 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.154016 Jun 10 13:13:17.606028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 487/1009] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:17 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:18.134966 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-417e4736-afe3-40a6-9cf9-40605506dc9e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:18.135886 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-417e4736-afe3-40a6-9cf9-40605506dc9e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:18.142621 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-417e4736-afe3-40a6-9cf9-40605506dc9e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:18.143066 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-417e4736-afe3-40a6-9cf9-40605506dc9e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:18.143830 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-417e4736-afe3-40a6-9cf9-40605506dc9e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:18.144260 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-417e4736-afe3-40a6-9cf9-40605506dc9e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:18.173520 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-417e4736-afe3-40a6-9cf9-40605506dc9e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042523 Jun 10 13:13:18.174080 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 523/1010] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:18 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:18.623145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d48c24dd-e4c9-4b74-84bc-320ac77d34c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:18.623709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d48c24dd-e4c9-4b74-84bc-320ac77d34c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:18.631466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d48c24dd-e4c9-4b74-84bc-320ac77d34c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:18.631905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d48c24dd-e4c9-4b74-84bc-320ac77d34c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:18.632655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d48c24dd-e4c9-4b74-84bc-320ac77d34c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:18.633117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d48c24dd-e4c9-4b74-84bc-320ac77d34c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:18.760056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d48c24dd-e4c9-4b74-84bc-320ac77d34c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:18.767440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d48c24dd-e4c9-4b74-84bc-320ac77d34c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148205 Jun 10 13:13:18.767992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 488/1011] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:18 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:19.185106 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2dda64ea-70d5-49bc-81fa-e1eaf1e83144 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:19.186023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2dda64ea-70d5-49bc-81fa-e1eaf1e83144 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:19.192485 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dda64ea-70d5-49bc-81fa-e1eaf1e83144 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:19.193023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dda64ea-70d5-49bc-81fa-e1eaf1e83144 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:19.201372 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dda64ea-70d5-49bc-81fa-e1eaf1e83144 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:19.201372 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dda64ea-70d5-49bc-81fa-e1eaf1e83144 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:19.222471 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2dda64ea-70d5-49bc-81fa-e1eaf1e83144 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041358 Jun 10 13:13:19.223016 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 524/1012] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:19 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:19.784142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7dbf0ccc-ee16-49c4-9097-258d772620e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:19.784749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7dbf0ccc-ee16-49c4-9097-258d772620e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:19.787180 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=52 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:19.791800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7dbf0ccc-ee16-49c4-9097-258d772620e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:19.792242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7dbf0ccc-ee16-49c4-9097-258d772620e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:19.793025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7dbf0ccc-ee16-49c4-9097-258d772620e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:19.793458 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7dbf0ccc-ee16-49c4-9097-258d772620e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:19.921905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7dbf0ccc-ee16-49c4-9097-258d772620e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:19.929896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7dbf0ccc-ee16-49c4-9097-258d772620e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149618 Jun 10 13:13:19.930450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 489/1013] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:19 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:20.233961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-52cc2af0-a7b8-4a71-bc82-346f1c551642 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:20.234862 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-52cc2af0-a7b8-4a71-bc82-346f1c551642 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:20.242129 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52cc2af0-a7b8-4a71-bc82-346f1c551642 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:20.242838 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52cc2af0-a7b8-4a71-bc82-346f1c551642 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:20.243587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52cc2af0-a7b8-4a71-bc82-346f1c551642 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:20.244015 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52cc2af0-a7b8-4a71-bc82-346f1c551642 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:20.271932 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-52cc2af0-a7b8-4a71-bc82-346f1c551642 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041995 Jun 10 13:13:20.272469 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 525/1014] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:20 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:20.947171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2d5743be-7be9-406e-9ca6-afd07c49d48b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:20.947806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2d5743be-7be9-406e-9ca6-afd07c49d48b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:20.954203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d5743be-7be9-406e-9ca6-afd07c49d48b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:20.954650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d5743be-7be9-406e-9ca6-afd07c49d48b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:20.955398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d5743be-7be9-406e-9ca6-afd07c49d48b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:20.955826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d5743be-7be9-406e-9ca6-afd07c49d48b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:21.087236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2d5743be-7be9-406e-9ca6-afd07c49d48b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:21.094600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2d5743be-7be9-406e-9ca6-afd07c49d48b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151813 Jun 10 13:13:21.095150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 490/1015] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:20 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:21.283646 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cfa71cc0-e686-437a-9943-e195373badf6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:21.284483 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cfa71cc0-e686-437a-9943-e195373badf6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:21.290877 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cfa71cc0-e686-437a-9943-e195373badf6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:21.291326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cfa71cc0-e686-437a-9943-e195373badf6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:21.292082 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cfa71cc0-e686-437a-9943-e195373badf6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:21.292513 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cfa71cc0-e686-437a-9943-e195373badf6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:21.321059 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cfa71cc0-e686-437a-9943-e195373badf6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041440 Jun 10 13:13:21.321605 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 526/1016] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:21 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:22.111522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ba59a194-f31c-413d-b108-64c7d573c6c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:22.112385 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ba59a194-f31c-413d-b108-64c7d573c6c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:22.118924 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba59a194-f31c-413d-b108-64c7d573c6c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:22.119376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba59a194-f31c-413d-b108-64c7d573c6c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:22.120134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba59a194-f31c-413d-b108-64c7d573c6c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:22.120567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba59a194-f31c-413d-b108-64c7d573c6c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:22.247805 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ba59a194-f31c-413d-b108-64c7d573c6c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:22.255207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ba59a194-f31c-413d-b108-64c7d573c6c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147559 Jun 10 13:13:22.255765 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 491/1017] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:22 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:22.332509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c80e5eec-17c7-4977-9604-2e4afcc00846 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:22.334054 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c80e5eec-17c7-4977-9604-2e4afcc00846 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:22.340691 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c80e5eec-17c7-4977-9604-2e4afcc00846 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:22.341181 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c80e5eec-17c7-4977-9604-2e4afcc00846 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:22.341948 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c80e5eec-17c7-4977-9604-2e4afcc00846 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:22.342389 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c80e5eec-17c7-4977-9604-2e4afcc00846 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:22.371693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c80e5eec-17c7-4977-9604-2e4afcc00846 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043148 Jun 10 13:13:22.372237 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 527/1018] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:22 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:23.271417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6a47c4ab-6094-4029-a8f7-4813b0f01ada tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:23.272019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6a47c4ab-6094-4029-a8f7-4813b0f01ada tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:23.278477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6a47c4ab-6094-4029-a8f7-4813b0f01ada tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:23.278949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6a47c4ab-6094-4029-a8f7-4813b0f01ada tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:23.279691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6a47c4ab-6094-4029-a8f7-4813b0f01ada tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:23.280114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6a47c4ab-6094-4029-a8f7-4813b0f01ada tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:23.383610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e45189ce-aed8-4488-9689-6c160048ba5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:23.387085 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e45189ce-aed8-4488-9689-6c160048ba5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:23.391428 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e45189ce-aed8-4488-9689-6c160048ba5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:23.391873 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e45189ce-aed8-4488-9689-6c160048ba5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:23.392632 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e45189ce-aed8-4488-9689-6c160048ba5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:23.393088 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e45189ce-aed8-4488-9689-6c160048ba5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:23.409500 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:23.413138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6a47c4ab-6094-4029-a8f7-4813b0f01ada tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:23.420806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6a47c4ab-6094-4029-a8f7-4813b0f01ada tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.153273 Jun 10 13:13:23.421381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 492/1019] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:23 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:23.422678 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e45189ce-aed8-4488-9689-6c160048ba5f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042997 Jun 10 13:13:23.423532 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 528/1020] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:23 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:13:24.434555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ef22fc34-e0a4-4148-b1ee-065b7ce25ea9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:24.435464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ef22fc34-e0a4-4148-b1ee-065b7ce25ea9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:24.438293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e550700c-62dc-442d-8169-68725e6aa000 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:24.438905 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e550700c-62dc-442d-8169-68725e6aa000 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:24.441949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ef22fc34-e0a4-4148-b1ee-065b7ce25ea9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:24.442390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ef22fc34-e0a4-4148-b1ee-065b7ce25ea9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:24.443732 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ef22fc34-e0a4-4148-b1ee-065b7ce25ea9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:24.444163 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ef22fc34-e0a4-4148-b1ee-065b7ce25ea9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:24.446326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e550700c-62dc-442d-8169-68725e6aa000 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:24.446775 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e550700c-62dc-442d-8169-68725e6aa000 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:24.448633 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e550700c-62dc-442d-8169-68725e6aa000 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:24.448633 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e550700c-62dc-442d-8169-68725e6aa000 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:24.472601 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ef22fc34-e0a4-4148-b1ee-065b7ce25ea9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041998 Jun 10 13:13:24.473184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 493/1021] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:24 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:24.578524 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e550700c-62dc-442d-8169-68725e6aa000 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:24.586079 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e550700c-62dc-442d-8169-68725e6aa000 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151545 Jun 10 13:13:24.586679 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 529/1022] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:24 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:13:25.484277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bb1e7e3f-2f13-4630-ad98-beae2ab57d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:25.485213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bb1e7e3f-2f13-4630-ad98-beae2ab57d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:25.491986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb1e7e3f-2f13-4630-ad98-beae2ab57d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:25.492433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb1e7e3f-2f13-4630-ad98-beae2ab57d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:25.493225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb1e7e3f-2f13-4630-ad98-beae2ab57d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:25.493659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb1e7e3f-2f13-4630-ad98-beae2ab57d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:25.523002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bb1e7e3f-2f13-4630-ad98-beae2ab57d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042683 Jun 10 13:13:25.523542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 494/1023] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:25 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:13:25.603064 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-750c198a-346f-413a-af98-7167e7e8dd45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:25.603607 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-750c198a-346f-413a-af98-7167e7e8dd45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:25.610039 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-750c198a-346f-413a-af98-7167e7e8dd45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:25.610484 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-750c198a-346f-413a-af98-7167e7e8dd45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:25.611242 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-750c198a-346f-413a-af98-7167e7e8dd45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:25.611671 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-750c198a-346f-413a-af98-7167e7e8dd45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:25.736275 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-750c198a-346f-413a-af98-7167e7e8dd45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:25.743958 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-750c198a-346f-413a-af98-7167e7e8dd45 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.144834 Jun 10 13:13:25.744518 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 530/1024] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:25 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:26.534454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e9b30119-1d25-4114-bee1-bf472034ad89 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:26.535359 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e9b30119-1d25-4114-bee1-bf472034ad89 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:26.542297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9b30119-1d25-4114-bee1-bf472034ad89 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:26.542736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9b30119-1d25-4114-bee1-bf472034ad89 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:26.543482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9b30119-1d25-4114-bee1-bf472034ad89 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:26.543909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9b30119-1d25-4114-bee1-bf472034ad89 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:26.573803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e9b30119-1d25-4114-bee1-bf472034ad89 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043328 Jun 10 13:13:26.574335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 495/1025] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:26 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:26.761372 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d7a5ad01-f2a1-45d8-ba42-eadd4aefc5ca tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:26.761988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d7a5ad01-f2a1-45d8-ba42-eadd4aefc5ca tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:26.768447 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d7a5ad01-f2a1-45d8-ba42-eadd4aefc5ca tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:26.768930 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d7a5ad01-f2a1-45d8-ba42-eadd4aefc5ca tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:26.769691 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d7a5ad01-f2a1-45d8-ba42-eadd4aefc5ca tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:26.770123 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d7a5ad01-f2a1-45d8-ba42-eadd4aefc5ca tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:26.900538 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d7a5ad01-f2a1-45d8-ba42-eadd4aefc5ca tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:26.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d7a5ad01-f2a1-45d8-ba42-eadd4aefc5ca tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150691 Jun 10 13:13:26.908839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 531/1026] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:26 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:27.585510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ff243355-13d0-4189-a965-03f2fca290b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:27.586354 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ff243355-13d0-4189-a965-03f2fca290b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:27.592818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff243355-13d0-4189-a965-03f2fca290b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:27.593266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff243355-13d0-4189-a965-03f2fca290b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:27.594018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff243355-13d0-4189-a965-03f2fca290b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:27.594450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff243355-13d0-4189-a965-03f2fca290b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:27.622666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ff243355-13d0-4189-a965-03f2fca290b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041251 Jun 10 13:13:27.623201 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 496/1027] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:27 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:27.925663 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2d11bac1-952f-44e9-891e-ad8e1315582c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:27.926195 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2d11bac1-952f-44e9-891e-ad8e1315582c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:27.933185 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d11bac1-952f-44e9-891e-ad8e1315582c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:27.933625 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d11bac1-952f-44e9-891e-ad8e1315582c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:27.934373 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d11bac1-952f-44e9-891e-ad8e1315582c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:27.934809 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d11bac1-952f-44e9-891e-ad8e1315582c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:28.061279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2d11bac1-952f-44e9-891e-ad8e1315582c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:28.069188 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2d11bac1-952f-44e9-891e-ad8e1315582c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147458 Jun 10 13:13:28.069766 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 532/1028] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:27 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:28.634260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-13aaa209-548e-4d38-8e3d-db7da1f37c06 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:28.635088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-13aaa209-548e-4d38-8e3d-db7da1f37c06 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:28.641324 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-13aaa209-548e-4d38-8e3d-db7da1f37c06 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:28.644783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-13aaa209-548e-4d38-8e3d-db7da1f37c06 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:28.644783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-13aaa209-548e-4d38-8e3d-db7da1f37c06 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:28.644783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-13aaa209-548e-4d38-8e3d-db7da1f37c06 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:28.671458 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-13aaa209-548e-4d38-8e3d-db7da1f37c06 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041185 Jun 10 13:13:28.672009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 497/1029] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:28 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:29.086614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-46c44f56-be85-4ec1-b3ad-0838af416799 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:29.087219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-46c44f56-be85-4ec1-b3ad-0838af416799 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:29.093604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46c44f56-be85-4ec1-b3ad-0838af416799 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:29.094044 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46c44f56-be85-4ec1-b3ad-0838af416799 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:29.094788 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46c44f56-be85-4ec1-b3ad-0838af416799 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:29.095212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46c44f56-be85-4ec1-b3ad-0838af416799 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:29.229481 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-46c44f56-be85-4ec1-b3ad-0838af416799 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:29.238216 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-46c44f56-be85-4ec1-b3ad-0838af416799 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.155952 Jun 10 13:13:29.238799 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 533/1030] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:29 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:29.683617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e638d1d7-78e3-457f-9a76-8407c9c80cf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:29.684527 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e638d1d7-78e3-457f-9a76-8407c9c80cf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:29.690978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e638d1d7-78e3-457f-9a76-8407c9c80cf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:29.691425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e638d1d7-78e3-457f-9a76-8407c9c80cf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:29.692179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e638d1d7-78e3-457f-9a76-8407c9c80cf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:29.692615 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e638d1d7-78e3-457f-9a76-8407c9c80cf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:29.720602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e638d1d7-78e3-457f-9a76-8407c9c80cf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041030 Jun 10 13:13:29.721161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 498/1031] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:29 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:30.255034 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4b22a2ee-38e4-4328-b8ce-b10edab824ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:30.255598 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4b22a2ee-38e4-4328-b8ce-b10edab824ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:30.262072 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b22a2ee-38e4-4328-b8ce-b10edab824ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:30.262511 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b22a2ee-38e4-4328-b8ce-b10edab824ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:30.263261 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b22a2ee-38e4-4328-b8ce-b10edab824ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:30.263688 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4b22a2ee-38e4-4328-b8ce-b10edab824ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:30.389498 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4b22a2ee-38e4-4328-b8ce-b10edab824ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:30.397444 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4b22a2ee-38e4-4328-b8ce-b10edab824ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146458 Jun 10 13:13:30.398004 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 534/1032] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:30 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:30.731573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d09a8543-5d50-458d-9179-542efe5ee40b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:30.732494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d09a8543-5d50-458d-9179-542efe5ee40b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:30.739244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d09a8543-5d50-458d-9179-542efe5ee40b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:30.739689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d09a8543-5d50-458d-9179-542efe5ee40b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:30.740450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d09a8543-5d50-458d-9179-542efe5ee40b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:30.740917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d09a8543-5d50-458d-9179-542efe5ee40b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:30.769079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d09a8543-5d50-458d-9179-542efe5ee40b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041483 Jun 10 13:13:30.769615 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 499/1033] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:30 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:31.414473 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-70601f83-424d-46aa-883d-680610f52fe2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:31.415023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-70601f83-424d-46aa-883d-680610f52fe2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:31.417335 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=60 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:31.422037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-70601f83-424d-46aa-883d-680610f52fe2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:31.422489 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-70601f83-424d-46aa-883d-680610f52fe2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:31.423261 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-70601f83-424d-46aa-883d-680610f52fe2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:31.423696 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-70601f83-424d-46aa-883d-680610f52fe2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:31.550360 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-70601f83-424d-46aa-883d-680610f52fe2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:31.558412 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-70601f83-424d-46aa-883d-680610f52fe2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147921 Jun 10 13:13:31.558995 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 535/1034] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:31 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:31.781255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-707a3e5b-3dbb-4a69-ab9c-abd793561054 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:31.782181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-707a3e5b-3dbb-4a69-ab9c-abd793561054 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:31.788900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-707a3e5b-3dbb-4a69-ab9c-abd793561054 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:31.789351 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-707a3e5b-3dbb-4a69-ab9c-abd793561054 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:31.790108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-707a3e5b-3dbb-4a69-ab9c-abd793561054 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:31.790541 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-707a3e5b-3dbb-4a69-ab9c-abd793561054 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:31.819446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-707a3e5b-3dbb-4a69-ab9c-abd793561054 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042067 Jun 10 13:13:31.820001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 500/1035] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:31 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:32.575404 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6615d4d5-89c0-4e8b-bbb6-f84b9143c8f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:32.576011 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6615d4d5-89c0-4e8b-bbb6-f84b9143c8f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:32.582509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6615d4d5-89c0-4e8b-bbb6-f84b9143c8f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:32.582956 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6615d4d5-89c0-4e8b-bbb6-f84b9143c8f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:32.584219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6615d4d5-89c0-4e8b-bbb6-f84b9143c8f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:32.584656 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6615d4d5-89c0-4e8b-bbb6-f84b9143c8f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:32.711410 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6615d4d5-89c0-4e8b-bbb6-f84b9143c8f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:32.718850 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6615d4d5-89c0-4e8b-bbb6-f84b9143c8f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147341 Jun 10 13:13:32.719412 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 536/1036] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:32 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:32.831010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f628e838-3a15-4fc6-bfc8-f0bd9d4363ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:32.831959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f628e838-3a15-4fc6-bfc8-f0bd9d4363ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:32.839041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f628e838-3a15-4fc6-bfc8-f0bd9d4363ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:32.839493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f628e838-3a15-4fc6-bfc8-f0bd9d4363ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:32.840256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f628e838-3a15-4fc6-bfc8-f0bd9d4363ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:32.840693 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f628e838-3a15-4fc6-bfc8-f0bd9d4363ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:32.869738 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f628e838-3a15-4fc6-bfc8-f0bd9d4363ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042694 Jun 10 13:13:32.870281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 501/1037] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:32 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:33.735312 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7bb83f45-da60-4923-9e81-f1f8dcae0fdb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:33.735906 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7bb83f45-da60-4923-9e81-f1f8dcae0fdb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:33.743133 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7bb83f45-da60-4923-9e81-f1f8dcae0fdb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:33.743572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7bb83f45-da60-4923-9e81-f1f8dcae0fdb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:33.744343 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7bb83f45-da60-4923-9e81-f1f8dcae0fdb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:33.744790 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7bb83f45-da60-4923-9e81-f1f8dcae0fdb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:33.869436 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7bb83f45-da60-4923-9e81-f1f8dcae0fdb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:33.876825 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7bb83f45-da60-4923-9e81-f1f8dcae0fdb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145315 Jun 10 13:13:33.877382 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 537/1038] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:33 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:33.881500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-315a4aaf-66ac-4a65-b4a7-730c3ee6411e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:33.882406 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-315a4aaf-66ac-4a65-b4a7-730c3ee6411e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:33.889148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-315a4aaf-66ac-4a65-b4a7-730c3ee6411e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:33.889600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-315a4aaf-66ac-4a65-b4a7-730c3ee6411e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:33.890395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-315a4aaf-66ac-4a65-b4a7-730c3ee6411e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:33.890832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-315a4aaf-66ac-4a65-b4a7-730c3ee6411e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:33.934498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-315a4aaf-66ac-4a65-b4a7-730c3ee6411e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057141 Jun 10 13:13:33.935062 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 502/1039] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:33 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:13:34.893118 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-dd58dd72-ba7b-4491-81f1-3151dbd7735d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:34.893723 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-dd58dd72-ba7b-4491-81f1-3151dbd7735d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:34.900638 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dd58dd72-ba7b-4491-81f1-3151dbd7735d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:34.901113 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dd58dd72-ba7b-4491-81f1-3151dbd7735d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:34.901868 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dd58dd72-ba7b-4491-81f1-3151dbd7735d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:34.902296 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dd58dd72-ba7b-4491-81f1-3151dbd7735d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:34.946089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1b65faf1-38c9-4dac-802a-61c54cae3e05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:34.947000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1b65faf1-38c9-4dac-802a-61c54cae3e05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:34.953513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b65faf1-38c9-4dac-802a-61c54cae3e05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:34.953964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b65faf1-38c9-4dac-802a-61c54cae3e05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:34.954725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b65faf1-38c9-4dac-802a-61c54cae3e05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:34.955158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b65faf1-38c9-4dac-802a-61c54cae3e05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:34.984538 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1b65faf1-38c9-4dac-802a-61c54cae3e05 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042401 Jun 10 13:13:34.985134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 503/1040] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:34 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:13:35.277527 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-dd58dd72-ba7b-4491-81f1-3151dbd7735d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:35.284976 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-dd58dd72-ba7b-4491-81f1-3151dbd7735d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.395689 Jun 10 13:13:35.285534 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 538/1041] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:34 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:35.996007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ae5bb709-e261-4dc9-b416-eac235ecf190 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:35.996962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ae5bb709-e261-4dc9-b416-eac235ecf190 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:36.003518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae5bb709-e261-4dc9-b416-eac235ecf190 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:36.003967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae5bb709-e261-4dc9-b416-eac235ecf190 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:36.004728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae5bb709-e261-4dc9-b416-eac235ecf190 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:36.005776 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae5bb709-e261-4dc9-b416-eac235ecf190 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:36.008117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:36.035116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ae5bb709-e261-4dc9-b416-eac235ecf190 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043133 Jun 10 13:13:36.035655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 504/1042] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:35 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:36.301306 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2f87a2db-5d98-4a8f-b39a-30775e2a0814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:36.302431 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2f87a2db-5d98-4a8f-b39a-30775e2a0814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:36.309125 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2f87a2db-5d98-4a8f-b39a-30775e2a0814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:36.309570 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2f87a2db-5d98-4a8f-b39a-30775e2a0814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:36.310330 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2f87a2db-5d98-4a8f-b39a-30775e2a0814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:36.310761 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2f87a2db-5d98-4a8f-b39a-30775e2a0814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:36.435746 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2f87a2db-5d98-4a8f-b39a-30775e2a0814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:36.443695 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2f87a2db-5d98-4a8f-b39a-30775e2a0814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146278 Jun 10 13:13:36.444253 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 539/1043] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:36 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:37.047059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-06fdd91c-b6d3-4a0b-ab06-e8003c347563 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:37.047878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-06fdd91c-b6d3-4a0b-ab06-e8003c347563 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:37.055492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06fdd91c-b6d3-4a0b-ab06-e8003c347563 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:37.055935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06fdd91c-b6d3-4a0b-ab06-e8003c347563 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:37.056690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06fdd91c-b6d3-4a0b-ab06-e8003c347563 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:37.057145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06fdd91c-b6d3-4a0b-ab06-e8003c347563 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:37.085742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-06fdd91c-b6d3-4a0b-ab06-e8003c347563 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042868 Jun 10 13:13:37.086284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 505/1044] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:37 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:37.229159 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:13:37.229408 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Fri Jun 10 13:13:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:13:37.460504 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d2e0950e-cbdc-4eda-9697-d7bb75e33969 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:37.461129 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d2e0950e-cbdc-4eda-9697-d7bb75e33969 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:37.467610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d2e0950e-cbdc-4eda-9697-d7bb75e33969 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:37.468054 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d2e0950e-cbdc-4eda-9697-d7bb75e33969 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:37.468836 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d2e0950e-cbdc-4eda-9697-d7bb75e33969 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:37.469274 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d2e0950e-cbdc-4eda-9697-d7bb75e33969 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:37.598519 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d2e0950e-cbdc-4eda-9697-d7bb75e33969 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:37.606341 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d2e0950e-cbdc-4eda-9697-d7bb75e33969 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149686 Jun 10 13:13:37.606896 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 540/1045] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:37 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:38.097216 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-596cbde1-b1ae-4f6a-bda4-2860c99c664c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:38.098120 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-596cbde1-b1ae-4f6a-bda4-2860c99c664c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:38.105236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-596cbde1-b1ae-4f6a-bda4-2860c99c664c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:38.105678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-596cbde1-b1ae-4f6a-bda4-2860c99c664c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:38.106427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-596cbde1-b1ae-4f6a-bda4-2860c99c664c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:38.106856 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-596cbde1-b1ae-4f6a-bda4-2860c99c664c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:38.134758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-596cbde1-b1ae-4f6a-bda4-2860c99c664c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041536 Jun 10 13:13:38.135302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 506/1046] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:38 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:38.623470 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-46c3dec4-c13a-4f17-91b8-0bdd876676a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:38.624048 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-46c3dec4-c13a-4f17-91b8-0bdd876676a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:38.631169 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46c3dec4-c13a-4f17-91b8-0bdd876676a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:38.631618 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46c3dec4-c13a-4f17-91b8-0bdd876676a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:38.632380 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46c3dec4-c13a-4f17-91b8-0bdd876676a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:38.632835 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46c3dec4-c13a-4f17-91b8-0bdd876676a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:38.760199 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-46c3dec4-c13a-4f17-91b8-0bdd876676a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:38.767628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-46c3dec4-c13a-4f17-91b8-0bdd876676a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148109 Jun 10 13:13:38.768183 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 541/1047] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:38 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:39.146297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-962b800b-bdd7-4e0e-ab63-376f565084b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:39.147205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-962b800b-bdd7-4e0e-ab63-376f565084b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:39.153578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-962b800b-bdd7-4e0e-ab63-376f565084b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:39.154024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-962b800b-bdd7-4e0e-ab63-376f565084b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:39.154771 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-962b800b-bdd7-4e0e-ab63-376f565084b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:39.155199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-962b800b-bdd7-4e0e-ab63-376f565084b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:39.184569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-962b800b-bdd7-4e0e-ab63-376f565084b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042512 Jun 10 13:13:39.185147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 507/1048] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:39 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:39.784660 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-bfe34c88-0e17-4fe9-93aa-9586d0ab1e2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:39.785331 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-bfe34c88-0e17-4fe9-93aa-9586d0ab1e2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:39.791402 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bfe34c88-0e17-4fe9-93aa-9586d0ab1e2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:39.791856 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bfe34c88-0e17-4fe9-93aa-9586d0ab1e2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:39.792613 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bfe34c88-0e17-4fe9-93aa-9586d0ab1e2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:39.793077 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bfe34c88-0e17-4fe9-93aa-9586d0ab1e2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:39.925078 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-bfe34c88-0e17-4fe9-93aa-9586d0ab1e2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:39.932686 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-bfe34c88-0e17-4fe9-93aa-9586d0ab1e2c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151987 Jun 10 13:13:39.933287 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 542/1049] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:39 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:40.196105 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-503ee6d7-6af6-496c-832d-ad302d1617a5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:40.197071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-503ee6d7-6af6-496c-832d-ad302d1617a5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:40.203421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-503ee6d7-6af6-496c-832d-ad302d1617a5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:40.203857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-503ee6d7-6af6-496c-832d-ad302d1617a5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:40.204637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-503ee6d7-6af6-496c-832d-ad302d1617a5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:40.205096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-503ee6d7-6af6-496c-832d-ad302d1617a5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:40.232545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-503ee6d7-6af6-496c-832d-ad302d1617a5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040982 Jun 10 13:13:40.233115 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 508/1050] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:40 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:40.553226 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:13:40.553413 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Fri Jun 10 13:13:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:13:40.954333 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5487a059-45f1-4a8a-ac56-7bfbad4fb275 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:40.955473 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5487a059-45f1-4a8a-ac56-7bfbad4fb275 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:40.963793 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5487a059-45f1-4a8a-ac56-7bfbad4fb275 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:40.964236 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5487a059-45f1-4a8a-ac56-7bfbad4fb275 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:40.965028 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5487a059-45f1-4a8a-ac56-7bfbad4fb275 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:40.965462 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5487a059-45f1-4a8a-ac56-7bfbad4fb275 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:41.093797 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5487a059-45f1-4a8a-ac56-7bfbad4fb275 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:41.101636 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5487a059-45f1-4a8a-ac56-7bfbad4fb275 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.153893 Jun 10 13:13:41.102185 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 543/1051] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:40 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:41.244144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4a1fb7ac-52c4-4137-a2d2-e8aee023a6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:41.244999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4a1fb7ac-52c4-4137-a2d2-e8aee023a6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:41.251543 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a1fb7ac-52c4-4137-a2d2-e8aee023a6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:41.251982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a1fb7ac-52c4-4137-a2d2-e8aee023a6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:41.252732 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a1fb7ac-52c4-4137-a2d2-e8aee023a6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:41.253181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a1fb7ac-52c4-4137-a2d2-e8aee023a6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:41.281970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4a1fb7ac-52c4-4137-a2d2-e8aee023a6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041850 Jun 10 13:13:41.282515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 509/1052] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:41 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:42.118673 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-87866748-01df-471d-9267-1dec0132b1d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:42.119312 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-87866748-01df-471d-9267-1dec0132b1d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:42.125737 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-87866748-01df-471d-9267-1dec0132b1d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:42.126176 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-87866748-01df-471d-9267-1dec0132b1d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:42.126925 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-87866748-01df-471d-9267-1dec0132b1d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:42.127351 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-87866748-01df-471d-9267-1dec0132b1d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:42.252436 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-87866748-01df-471d-9267-1dec0132b1d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:42.260604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-87866748-01df-471d-9267-1dec0132b1d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145712 Jun 10 13:13:42.261124 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 544/1053] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:42 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:42.292845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e46782ac-cb98-41c2-82bd-edabe81d585d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:42.293768 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e46782ac-cb98-41c2-82bd-edabe81d585d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:42.300187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e46782ac-cb98-41c2-82bd-edabe81d585d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:42.300639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e46782ac-cb98-41c2-82bd-edabe81d585d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:42.301424 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e46782ac-cb98-41c2-82bd-edabe81d585d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:42.302393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e46782ac-cb98-41c2-82bd-edabe81d585d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:42.330988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e46782ac-cb98-41c2-82bd-edabe81d585d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042102 Jun 10 13:13:42.331542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 510/1054] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:42 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:13:43.277693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a25bf49a-d627-4753-be46-c21800260e88 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:43.278320 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a25bf49a-d627-4753-be46-c21800260e88 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:43.285340 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a25bf49a-d627-4753-be46-c21800260e88 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:43.285786 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a25bf49a-d627-4753-be46-c21800260e88 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:43.287245 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a25bf49a-d627-4753-be46-c21800260e88 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:43.287685 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a25bf49a-d627-4753-be46-c21800260e88 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:43.292103 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:43.342744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c1a60954-4109-4ff6-97d6-226e57b15640 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:43.343670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c1a60954-4109-4ff6-97d6-226e57b15640 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:43.350744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1a60954-4109-4ff6-97d6-226e57b15640 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:43.351189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1a60954-4109-4ff6-97d6-226e57b15640 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:43.351965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1a60954-4109-4ff6-97d6-226e57b15640 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:43.352402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1a60954-4109-4ff6-97d6-226e57b15640 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:43.382781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c1a60954-4109-4ff6-97d6-226e57b15640 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044082 Jun 10 13:13:43.383360 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 511/1055] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:43 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:43.432227 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a25bf49a-d627-4753-be46-c21800260e88 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:43.439951 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a25bf49a-d627-4753-be46-c21800260e88 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.166223 Jun 10 13:13:43.443292 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 545/1056] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:43 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:13:44.394427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-51bb0f1c-df70-43e0-a53b-caef59136138 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:44.395340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-51bb0f1c-df70-43e0-a53b-caef59136138 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:44.401769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-51bb0f1c-df70-43e0-a53b-caef59136138 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:44.402212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-51bb0f1c-df70-43e0-a53b-caef59136138 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:44.402962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-51bb0f1c-df70-43e0-a53b-caef59136138 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:44.403392 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-51bb0f1c-df70-43e0-a53b-caef59136138 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:44.432022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-51bb0f1c-df70-43e0-a53b-caef59136138 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041575 Jun 10 13:13:44.432551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 512/1057] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:44 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:44.459356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2929e229-c9ce-4359-9401-fb133c3b5368 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:44.459959 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2929e229-c9ce-4359-9401-fb133c3b5368 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:44.466444 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2929e229-c9ce-4359-9401-fb133c3b5368 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:44.466889 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2929e229-c9ce-4359-9401-fb133c3b5368 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:44.467640 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2929e229-c9ce-4359-9401-fb133c3b5368 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:44.468072 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2929e229-c9ce-4359-9401-fb133c3b5368 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:44.603377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2929e229-c9ce-4359-9401-fb133c3b5368 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:44.610781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2929e229-c9ce-4359-9401-fb133c3b5368 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.155224 Jun 10 13:13:44.611331 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 546/1058] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:44 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:45.444127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e9e5b306-cb8e-4e07-958e-6b3c64c9a78b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:45.445065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e9e5b306-cb8e-4e07-958e-6b3c64c9a78b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:45.451475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9e5b306-cb8e-4e07-958e-6b3c64c9a78b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:45.451924 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9e5b306-cb8e-4e07-958e-6b3c64c9a78b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:45.452681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9e5b306-cb8e-4e07-958e-6b3c64c9a78b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:45.453140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9e5b306-cb8e-4e07-958e-6b3c64c9a78b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:45.480579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e9e5b306-cb8e-4e07-958e-6b3c64c9a78b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041002 Jun 10 13:13:45.481156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 513/1059] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:45 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:45.627857 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d3fb1fa5-ea7c-4a39-97f9-178683ac4a5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:45.628392 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d3fb1fa5-ea7c-4a39-97f9-178683ac4a5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:45.635628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d3fb1fa5-ea7c-4a39-97f9-178683ac4a5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:45.636073 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d3fb1fa5-ea7c-4a39-97f9-178683ac4a5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:45.636839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d3fb1fa5-ea7c-4a39-97f9-178683ac4a5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:45.637271 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d3fb1fa5-ea7c-4a39-97f9-178683ac4a5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:45.763430 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d3fb1fa5-ea7c-4a39-97f9-178683ac4a5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:45.770768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d3fb1fa5-ea7c-4a39-97f9-178683ac4a5f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146756 Jun 10 13:13:45.771315 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 547/1060] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:45 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:46.492481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-72a565e9-2f5f-444f-9787-4c4311463fec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:46.493399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-72a565e9-2f5f-444f-9787-4c4311463fec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:46.499783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72a565e9-2f5f-444f-9787-4c4311463fec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:46.500225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72a565e9-2f5f-444f-9787-4c4311463fec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:46.501009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72a565e9-2f5f-444f-9787-4c4311463fec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:46.501447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72a565e9-2f5f-444f-9787-4c4311463fec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:46.529412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-72a565e9-2f5f-444f-9787-4c4311463fec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041394 Jun 10 13:13:46.530147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 514/1061] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:46 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:46.787901 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-483f37b6-eb50-4fb2-8986-930eb690f24b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:46.788501 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-483f37b6-eb50-4fb2-8986-930eb690f24b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:46.795116 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-483f37b6-eb50-4fb2-8986-930eb690f24b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:46.795557 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-483f37b6-eb50-4fb2-8986-930eb690f24b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:46.796302 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-483f37b6-eb50-4fb2-8986-930eb690f24b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:46.796730 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-483f37b6-eb50-4fb2-8986-930eb690f24b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:46.926127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-483f37b6-eb50-4fb2-8986-930eb690f24b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:46.933604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-483f37b6-eb50-4fb2-8986-930eb690f24b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150074 Jun 10 13:13:46.934151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 548/1062] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:46 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:47.540393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1467ce4d-b2a5-4e36-9e6d-fbed876db2a3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:47.541338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1467ce4d-b2a5-4e36-9e6d-fbed876db2a3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:47.547763 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1467ce4d-b2a5-4e36-9e6d-fbed876db2a3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:47.548207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1467ce4d-b2a5-4e36-9e6d-fbed876db2a3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:47.548993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1467ce4d-b2a5-4e36-9e6d-fbed876db2a3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:47.549426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1467ce4d-b2a5-4e36-9e6d-fbed876db2a3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:47.578451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1467ce4d-b2a5-4e36-9e6d-fbed876db2a3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042048 Jun 10 13:13:47.578990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 515/1063] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:47 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:47.949957 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0f6cd5d0-885d-4a5a-8df1-38892e965225 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:47.950567 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0f6cd5d0-885d-4a5a-8df1-38892e965225 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:47.957117 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f6cd5d0-885d-4a5a-8df1-38892e965225 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:47.957583 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f6cd5d0-885d-4a5a-8df1-38892e965225 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:47.958323 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f6cd5d0-885d-4a5a-8df1-38892e965225 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:47.958759 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f6cd5d0-885d-4a5a-8df1-38892e965225 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:48.084922 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0f6cd5d0-885d-4a5a-8df1-38892e965225 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:48.092280 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0f6cd5d0-885d-4a5a-8df1-38892e965225 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146164 Jun 10 13:13:48.092860 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 549/1064] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:47 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:48.590571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-be6ba59f-d9ee-40d9-85f1-6b3a97d90fde tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:48.591433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-be6ba59f-d9ee-40d9-85f1-6b3a97d90fde tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:48.597946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be6ba59f-d9ee-40d9-85f1-6b3a97d90fde tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:48.598390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be6ba59f-d9ee-40d9-85f1-6b3a97d90fde tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:48.599710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be6ba59f-d9ee-40d9-85f1-6b3a97d90fde tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:48.600142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be6ba59f-d9ee-40d9-85f1-6b3a97d90fde tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:48.628967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-be6ba59f-d9ee-40d9-85f1-6b3a97d90fde tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042482 Jun 10 13:13:48.629519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 516/1065] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:48 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:49.109821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ecc57564-5f58-4372-9811-2522e04a0932 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:49.110478 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ecc57564-5f58-4372-9811-2522e04a0932 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:49.117696 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ecc57564-5f58-4372-9811-2522e04a0932 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:49.118141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ecc57564-5f58-4372-9811-2522e04a0932 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:49.118903 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ecc57564-5f58-4372-9811-2522e04a0932 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:49.119334 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ecc57564-5f58-4372-9811-2522e04a0932 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:49.252619 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ecc57564-5f58-4372-9811-2522e04a0932 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:49.260631 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ecc57564-5f58-4372-9811-2522e04a0932 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.154726 Jun 10 13:13:49.261218 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 550/1066] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:49 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:49.640039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b16ebf3d-07fa-4cd1-bc2f-4153e20c0e45 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:49.640975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b16ebf3d-07fa-4cd1-bc2f-4153e20c0e45 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:49.647489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b16ebf3d-07fa-4cd1-bc2f-4153e20c0e45 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:49.647936 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b16ebf3d-07fa-4cd1-bc2f-4153e20c0e45 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:49.648689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b16ebf3d-07fa-4cd1-bc2f-4153e20c0e45 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:49.649151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b16ebf3d-07fa-4cd1-bc2f-4153e20c0e45 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:49.677882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b16ebf3d-07fa-4cd1-bc2f-4153e20c0e45 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041857 Jun 10 13:13:49.678419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 517/1067] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:49 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:50.277952 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2c121f0f-61b9-4bb6-a5c0-343ad581af56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:50.278490 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2c121f0f-61b9-4bb6-a5c0-343ad581af56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:50.284971 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c121f0f-61b9-4bb6-a5c0-343ad581af56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:50.285413 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c121f0f-61b9-4bb6-a5c0-343ad581af56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:50.286160 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c121f0f-61b9-4bb6-a5c0-343ad581af56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:50.286588 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c121f0f-61b9-4bb6-a5c0-343ad581af56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:50.412586 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2c121f0f-61b9-4bb6-a5c0-343ad581af56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:50.420265 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2c121f0f-61b9-4bb6-a5c0-343ad581af56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146233 Jun 10 13:13:50.420846 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 551/1068] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:50 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:50.689754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-58bec576-898c-49c1-a8b1-6ea2bfa7f183 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:50.690580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-58bec576-898c-49c1-a8b1-6ea2bfa7f183 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:50.693518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=58 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:50.698213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58bec576-898c-49c1-a8b1-6ea2bfa7f183 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:50.698668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58bec576-898c-49c1-a8b1-6ea2bfa7f183 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:50.699428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58bec576-898c-49c1-a8b1-6ea2bfa7f183 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:50.699871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58bec576-898c-49c1-a8b1-6ea2bfa7f183 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:50.728531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-58bec576-898c-49c1-a8b1-6ea2bfa7f183 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042814 Jun 10 13:13:50.729122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 518/1069] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:50 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:51.437349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-588b3cf0-47ab-4aa1-97d2-453623400281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:51.437958 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-588b3cf0-47ab-4aa1-97d2-453623400281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:51.444602 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-588b3cf0-47ab-4aa1-97d2-453623400281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:51.445090 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-588b3cf0-47ab-4aa1-97d2-453623400281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:51.445846 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-588b3cf0-47ab-4aa1-97d2-453623400281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:51.446278 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-588b3cf0-47ab-4aa1-97d2-453623400281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:51.577403 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-588b3cf0-47ab-4aa1-97d2-453623400281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:51.585396 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-588b3cf0-47ab-4aa1-97d2-453623400281 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151863 Jun 10 13:13:51.586023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 552/1070] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:51 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:51.739998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4cc34768-0b2c-4df3-b502-cfe35897569b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:51.740970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4cc34768-0b2c-4df3-b502-cfe35897569b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:51.748074 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4cc34768-0b2c-4df3-b502-cfe35897569b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:51.748522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4cc34768-0b2c-4df3-b502-cfe35897569b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:51.749321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4cc34768-0b2c-4df3-b502-cfe35897569b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:51.749757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4cc34768-0b2c-4df3-b502-cfe35897569b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:51.779935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4cc34768-0b2c-4df3-b502-cfe35897569b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044324 Jun 10 13:13:51.780502 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 519/1071] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:51 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:52.603369 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cff3823b-2669-4108-a78a-11b274f8232e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:52.603904 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cff3823b-2669-4108-a78a-11b274f8232e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:52.610366 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cff3823b-2669-4108-a78a-11b274f8232e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:52.610812 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cff3823b-2669-4108-a78a-11b274f8232e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:52.611570 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cff3823b-2669-4108-a78a-11b274f8232e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:52.612252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cff3823b-2669-4108-a78a-11b274f8232e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:52.737793 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cff3823b-2669-4108-a78a-11b274f8232e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:52.745796 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cff3823b-2669-4108-a78a-11b274f8232e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147207 Jun 10 13:13:52.746354 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 553/1072] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:52 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:52.791834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-25840132-eb54-492b-84b7-ce70887bcedd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:52.792722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-25840132-eb54-492b-84b7-ce70887bcedd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:52.799193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25840132-eb54-492b-84b7-ce70887bcedd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:52.799631 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25840132-eb54-492b-84b7-ce70887bcedd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:52.800380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25840132-eb54-492b-84b7-ce70887bcedd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:52.800832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25840132-eb54-492b-84b7-ce70887bcedd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:52.824961 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:52.829261 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-25840132-eb54-492b-84b7-ce70887bcedd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041377 Jun 10 13:13:52.829803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 520/1073] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:52 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:53.762628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-72801447-d3b7-40c9-8d3c-0145a824a404 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:53.763435 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-72801447-d3b7-40c9-8d3c-0145a824a404 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:53.769830 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-72801447-d3b7-40c9-8d3c-0145a824a404 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:53.770270 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-72801447-d3b7-40c9-8d3c-0145a824a404 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:53.771110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-72801447-d3b7-40c9-8d3c-0145a824a404 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:53.771533 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-72801447-d3b7-40c9-8d3c-0145a824a404 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:53.841301 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6c219e5b-eb10-4f59-b8fa-ff915605e157 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:53.842210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6c219e5b-eb10-4f59-b8fa-ff915605e157 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:53.852672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c219e5b-eb10-4f59-b8fa-ff915605e157 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:53.852672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c219e5b-eb10-4f59-b8fa-ff915605e157 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:53.852672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c219e5b-eb10-4f59-b8fa-ff915605e157 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:53.852672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c219e5b-eb10-4f59-b8fa-ff915605e157 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:53.881690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6c219e5b-eb10-4f59-b8fa-ff915605e157 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044434 Jun 10 13:13:53.882247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 521/1074] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:53 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:13:53.900140 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-72801447-d3b7-40c9-8d3c-0145a824a404 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:53.907488 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-72801447-d3b7-40c9-8d3c-0145a824a404 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148694 Jun 10 13:13:53.908034 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 554/1075] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:53 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:54.893426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ca1462d8-209f-4c38-b4f4-fc9ed583e389 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:54.894364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ca1462d8-209f-4c38-b4f4-fc9ed583e389 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:54.901599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca1462d8-209f-4c38-b4f4-fc9ed583e389 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:54.902077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca1462d8-209f-4c38-b4f4-fc9ed583e389 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:54.902848 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca1462d8-209f-4c38-b4f4-fc9ed583e389 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:54.903292 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca1462d8-209f-4c38-b4f4-fc9ed583e389 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:54.924632 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-735a85f2-9295-4d9c-8d75-3117d2b50fe3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:54.925274 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-735a85f2-9295-4d9c-8d75-3117d2b50fe3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:54.931524 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ca1462d8-209f-4c38-b4f4-fc9ed583e389 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042114 Jun 10 13:13:54.931699 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-735a85f2-9295-4d9c-8d75-3117d2b50fe3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:54.931994 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-735a85f2-9295-4d9c-8d75-3117d2b50fe3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:54.932099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 522/1076] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:54 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:54.932758 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-735a85f2-9295-4d9c-8d75-3117d2b50fe3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:54.933211 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-735a85f2-9295-4d9c-8d75-3117d2b50fe3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:55.064328 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-735a85f2-9295-4d9c-8d75-3117d2b50fe3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:55.072341 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-735a85f2-9295-4d9c-8d75-3117d2b50fe3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151600 Jun 10 13:13:55.072922 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 555/1077] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:54 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:13:55.217806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:13:55.218034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:13:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:13:55.943425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9c78329b-f055-4b49-87b2-2d0c337c3d5c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:55.944339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9c78329b-f055-4b49-87b2-2d0c337c3d5c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:55.950769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c78329b-f055-4b49-87b2-2d0c337c3d5c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:55.951216 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c78329b-f055-4b49-87b2-2d0c337c3d5c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:55.951979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c78329b-f055-4b49-87b2-2d0c337c3d5c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:55.952417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c78329b-f055-4b49-87b2-2d0c337c3d5c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:55.980968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9c78329b-f055-4b49-87b2-2d0c337c3d5c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041604 Jun 10 13:13:55.981507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 523/1078] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:55 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:56.089552 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5a531c8e-fb31-41dc-9803-e5faba70d604 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:56.090153 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5a531c8e-fb31-41dc-9803-e5faba70d604 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:56.096584 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a531c8e-fb31-41dc-9803-e5faba70d604 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:56.097055 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a531c8e-fb31-41dc-9803-e5faba70d604 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:56.097805 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a531c8e-fb31-41dc-9803-e5faba70d604 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:56.098232 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a531c8e-fb31-41dc-9803-e5faba70d604 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:56.225587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5a531c8e-fb31-41dc-9803-e5faba70d604 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:56.233003 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5a531c8e-fb31-41dc-9803-e5faba70d604 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147226 Jun 10 13:13:56.233562 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 556/1079] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:56 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:56.992223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c346f943-18cd-4911-8e2d-4e0abb08f04d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:56.993176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c346f943-18cd-4911-8e2d-4e0abb08f04d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:56.999658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c346f943-18cd-4911-8e2d-4e0abb08f04d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:57.000108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c346f943-18cd-4911-8e2d-4e0abb08f04d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:57.000894 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c346f943-18cd-4911-8e2d-4e0abb08f04d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:57.001335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c346f943-18cd-4911-8e2d-4e0abb08f04d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:57.030012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c346f943-18cd-4911-8e2d-4e0abb08f04d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041852 Jun 10 13:13:57.030546 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 524/1080] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:56 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:57.250951 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ff3dae92-d522-4a9f-ab5f-3d9be395650f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:57.251509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ff3dae92-d522-4a9f-ab5f-3d9be395650f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:57.258600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ff3dae92-d522-4a9f-ab5f-3d9be395650f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:57.259043 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ff3dae92-d522-4a9f-ab5f-3d9be395650f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:57.259789 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ff3dae92-d522-4a9f-ab5f-3d9be395650f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:57.260215 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ff3dae92-d522-4a9f-ab5f-3d9be395650f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:57.390253 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ff3dae92-d522-4a9f-ab5f-3d9be395650f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:57.397743 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ff3dae92-d522-4a9f-ab5f-3d9be395650f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151260 Jun 10 13:13:57.398303 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 557/1081] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:57 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:58.041678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-006c69da-d466-495d-ade7-cb86bc664834 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:58.042582 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-006c69da-d466-495d-ade7-cb86bc664834 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:58.049044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-006c69da-d466-495d-ade7-cb86bc664834 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:58.049495 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-006c69da-d466-495d-ade7-cb86bc664834 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:58.050252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-006c69da-d466-495d-ade7-cb86bc664834 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:58.050686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-006c69da-d466-495d-ade7-cb86bc664834 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:58.078747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-006c69da-d466-495d-ade7-cb86bc664834 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041071 Jun 10 13:13:58.079280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 525/1082] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:58 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:58.415359 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f8296c9b-69f7-47dd-a775-a3b36e75292f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:58.415962 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f8296c9b-69f7-47dd-a775-a3b36e75292f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:58.422423 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f8296c9b-69f7-47dd-a775-a3b36e75292f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:58.426224 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f8296c9b-69f7-47dd-a775-a3b36e75292f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:58.426224 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f8296c9b-69f7-47dd-a775-a3b36e75292f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:58.426224 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f8296c9b-69f7-47dd-a775-a3b36e75292f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:58.552695 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f8296c9b-69f7-47dd-a775-a3b36e75292f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:58.560279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f8296c9b-69f7-47dd-a775-a3b36e75292f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149450 Jun 10 13:13:58.560861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 558/1083] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:58 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:13:59.090017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-323b38f4-599a-4b20-a389-789800bb62b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:59.090941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-323b38f4-599a-4b20-a389-789800bb62b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:59.097463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-323b38f4-599a-4b20-a389-789800bb62b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:59.097913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-323b38f4-599a-4b20-a389-789800bb62b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:59.098703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-323b38f4-599a-4b20-a389-789800bb62b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:59.099142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-323b38f4-599a-4b20-a389-789800bb62b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:59.128701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-323b38f4-599a-4b20-a389-789800bb62b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042935 Jun 10 13:13:59.129298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 526/1084] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:13:59 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:13:59.578074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1e594937-a9b5-4798-81e1-27dea2c34840 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:13:59.578681 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1e594937-a9b5-4798-81e1-27dea2c34840 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:13:59.585619 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1e594937-a9b5-4798-81e1-27dea2c34840 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:59.586063 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1e594937-a9b5-4798-81e1-27dea2c34840 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:59.586812 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1e594937-a9b5-4798-81e1-27dea2c34840 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:13:59.587242 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1e594937-a9b5-4798-81e1-27dea2c34840 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:13:59.713964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1e594937-a9b5-4798-81e1-27dea2c34840 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:13:59.721374 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1e594937-a9b5-4798-81e1-27dea2c34840 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147349 Jun 10 13:13:59.721934 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 559/1085] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:13:59 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:00.140712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f84ae1b0-40b8-41dd-aab3-9ce06933b819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:00.141680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f84ae1b0-40b8-41dd-aab3-9ce06933b819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:00.148742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f84ae1b0-40b8-41dd-aab3-9ce06933b819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:00.149252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f84ae1b0-40b8-41dd-aab3-9ce06933b819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:00.150014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f84ae1b0-40b8-41dd-aab3-9ce06933b819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:00.150447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f84ae1b0-40b8-41dd-aab3-9ce06933b819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:00.177963 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f84ae1b0-40b8-41dd-aab3-9ce06933b819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041211 Jun 10 13:14:00.178501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 527/1086] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:00 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:00.739544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-96ff6150-b99f-46cf-8f75-54fb3317c633 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:00.740748 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-96ff6150-b99f-46cf-8f75-54fb3317c633 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:00.747325 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-96ff6150-b99f-46cf-8f75-54fb3317c633 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:00.747765 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-96ff6150-b99f-46cf-8f75-54fb3317c633 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:00.748532 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-96ff6150-b99f-46cf-8f75-54fb3317c633 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:00.748968 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-96ff6150-b99f-46cf-8f75-54fb3317c633 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:00.878885 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-96ff6150-b99f-46cf-8f75-54fb3317c633 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:00.886464 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-96ff6150-b99f-46cf-8f75-54fb3317c633 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151146 Jun 10 13:14:00.887020 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 560/1087] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:00 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:01.189856 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-912cd313-3331-46da-b78e-63bea3f0c96b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:01.190676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-912cd313-3331-46da-b78e-63bea3f0c96b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:01.196600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-912cd313-3331-46da-b78e-63bea3f0c96b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:01.197082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-912cd313-3331-46da-b78e-63bea3f0c96b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:01.197833 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-912cd313-3331-46da-b78e-63bea3f0c96b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:01.198260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-912cd313-3331-46da-b78e-63bea3f0c96b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:01.226846 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-912cd313-3331-46da-b78e-63bea3f0c96b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041203 Jun 10 13:14:01.227378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 528/1088] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:01 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:01.903796 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cc41c655-66ee-4077-b3c1-7f30e5acfaf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:01.904396 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cc41c655-66ee-4077-b3c1-7f30e5acfaf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:01.906751 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=52 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:01.911445 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc41c655-66ee-4077-b3c1-7f30e5acfaf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:01.911936 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc41c655-66ee-4077-b3c1-7f30e5acfaf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:01.912685 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc41c655-66ee-4077-b3c1-7f30e5acfaf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:01.913138 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc41c655-66ee-4077-b3c1-7f30e5acfaf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:02.048630 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cc41c655-66ee-4077-b3c1-7f30e5acfaf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:02.052639 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:02.056819 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cc41c655-66ee-4077-b3c1-7f30e5acfaf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.156856 Jun 10 13:14:02.057376 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 561/1089] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:01 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:02.238720 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fc9c352c-ba0d-404a-8e9b-18b2b5e74d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:02.240203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fc9c352c-ba0d-404a-8e9b-18b2b5e74d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:02.246684 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fc9c352c-ba0d-404a-8e9b-18b2b5e74d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:02.247128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fc9c352c-ba0d-404a-8e9b-18b2b5e74d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:02.247878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fc9c352c-ba0d-404a-8e9b-18b2b5e74d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:02.248306 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fc9c352c-ba0d-404a-8e9b-18b2b5e74d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:02.276789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fc9c352c-ba0d-404a-8e9b-18b2b5e74d37 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042197 Jun 10 13:14:02.277346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 529/1090] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:02 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:03.074416 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2204f2ba-2d71-4276-b33e-b9d6cb99425e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:03.075010 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2204f2ba-2d71-4276-b33e-b9d6cb99425e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:03.081589 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2204f2ba-2d71-4276-b33e-b9d6cb99425e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:03.082035 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2204f2ba-2d71-4276-b33e-b9d6cb99425e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:03.082780 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2204f2ba-2d71-4276-b33e-b9d6cb99425e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:03.083211 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2204f2ba-2d71-4276-b33e-b9d6cb99425e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:03.213917 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2204f2ba-2d71-4276-b33e-b9d6cb99425e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:03.221492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2204f2ba-2d71-4276-b33e-b9d6cb99425e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151305 Jun 10 13:14:03.222058 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 562/1091] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:03 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:03.288061 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b374fe85-c0d1-4e6a-805a-d32033036ac0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:03.289007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b374fe85-c0d1-4e6a-805a-d32033036ac0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:03.295494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b374fe85-c0d1-4e6a-805a-d32033036ac0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:03.295934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b374fe85-c0d1-4e6a-805a-d32033036ac0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:03.296689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b374fe85-c0d1-4e6a-805a-d32033036ac0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:03.297147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b374fe85-c0d1-4e6a-805a-d32033036ac0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:03.326177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b374fe85-c0d1-4e6a-805a-d32033036ac0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042078 Jun 10 13:14:03.326661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 530/1092] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:03 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:04.237820 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a3539a22-1f31-4e9f-b051-05cb6fa0afdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:04.238418 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a3539a22-1f31-4e9f-b051-05cb6fa0afdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:04.244870 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3539a22-1f31-4e9f-b051-05cb6fa0afdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:04.245313 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3539a22-1f31-4e9f-b051-05cb6fa0afdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:04.246058 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3539a22-1f31-4e9f-b051-05cb6fa0afdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:04.246482 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a3539a22-1f31-4e9f-b051-05cb6fa0afdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:04.339262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0b8e76dc-d770-4938-acf9-d4deb02f6a00 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:04.340252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0b8e76dc-d770-4938-acf9-d4deb02f6a00 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:04.346867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b8e76dc-d770-4938-acf9-d4deb02f6a00 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:04.347318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b8e76dc-d770-4938-acf9-d4deb02f6a00 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:04.348083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b8e76dc-d770-4938-acf9-d4deb02f6a00 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:04.348519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b8e76dc-d770-4938-acf9-d4deb02f6a00 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:04.374616 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a3539a22-1f31-4e9f-b051-05cb6fa0afdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:04.378897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0b8e76dc-d770-4938-acf9-d4deb02f6a00 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043721 Jun 10 13:14:04.379472 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 531/1093] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:04 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:04.382024 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a3539a22-1f31-4e9f-b051-05cb6fa0afdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148312 Jun 10 13:14:04.382577 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 563/1094] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:04 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:05.390376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b56a439f-1bf6-4c77-8ee6-80200441c109 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:05.391285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b56a439f-1bf6-4c77-8ee6-80200441c109 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:05.397755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b56a439f-1bf6-4c77-8ee6-80200441c109 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:05.398202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b56a439f-1bf6-4c77-8ee6-80200441c109 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:05.398531 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-49c2761a-2aa3-4151-a389-8e8871ddb1f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:05.398952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b56a439f-1bf6-4c77-8ee6-80200441c109 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:05.399133 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-49c2761a-2aa3-4151-a389-8e8871ddb1f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:05.399365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b56a439f-1bf6-4c77-8ee6-80200441c109 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:05.405623 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-49c2761a-2aa3-4151-a389-8e8871ddb1f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:05.406063 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-49c2761a-2aa3-4151-a389-8e8871ddb1f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:05.406822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-49c2761a-2aa3-4151-a389-8e8871ddb1f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:05.407244 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-49c2761a-2aa3-4151-a389-8e8871ddb1f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:05.428560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b56a439f-1bf6-4c77-8ee6-80200441c109 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042106 Jun 10 13:14:05.429139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 532/1095] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:05 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:05.535398 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-49c2761a-2aa3-4151-a389-8e8871ddb1f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:05.543286 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-49c2761a-2aa3-4151-a389-8e8871ddb1f2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148593 Jun 10 13:14:05.543838 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 564/1096] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:05 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:06.440179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-56eecc9e-bf98-4f84-a64b-c969441bd0e7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:06.441150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-56eecc9e-bf98-4f84-a64b-c969441bd0e7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:06.448140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-56eecc9e-bf98-4f84-a64b-c969441bd0e7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:06.448588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-56eecc9e-bf98-4f84-a64b-c969441bd0e7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:06.449378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-56eecc9e-bf98-4f84-a64b-c969441bd0e7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:06.449814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-56eecc9e-bf98-4f84-a64b-c969441bd0e7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:06.477485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-56eecc9e-bf98-4f84-a64b-c969441bd0e7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041337 Jun 10 13:14:06.478010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 533/1097] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:06 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:06.560640 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-864066a9-1ccb-4490-a7be-3ff4be00aa38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:06.561286 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-864066a9-1ccb-4490-a7be-3ff4be00aa38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:06.568361 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-864066a9-1ccb-4490-a7be-3ff4be00aa38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:06.568830 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-864066a9-1ccb-4490-a7be-3ff4be00aa38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:06.569593 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-864066a9-1ccb-4490-a7be-3ff4be00aa38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:06.570027 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-864066a9-1ccb-4490-a7be-3ff4be00aa38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:06.696194 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-864066a9-1ccb-4490-a7be-3ff4be00aa38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:06.703586 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-864066a9-1ccb-4490-a7be-3ff4be00aa38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146788 Jun 10 13:14:06.704146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 565/1098] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:06 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:07.489339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-810f72ce-2fe2-494b-abb7-a1c4b6bdab91 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:07.490801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-810f72ce-2fe2-494b-abb7-a1c4b6bdab91 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:07.497277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-810f72ce-2fe2-494b-abb7-a1c4b6bdab91 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:07.497720 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-810f72ce-2fe2-494b-abb7-a1c4b6bdab91 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:07.498474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-810f72ce-2fe2-494b-abb7-a1c4b6bdab91 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:07.498903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-810f72ce-2fe2-494b-abb7-a1c4b6bdab91 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:07.526700 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-810f72ce-2fe2-494b-abb7-a1c4b6bdab91 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041594 Jun 10 13:14:07.527169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 534/1099] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:07 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:07.720577 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7d16b44a-38af-44b9-89a4-68b354695a3d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:07.721207 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7d16b44a-38af-44b9-89a4-68b354695a3d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:07.728272 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7d16b44a-38af-44b9-89a4-68b354695a3d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:07.728715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7d16b44a-38af-44b9-89a4-68b354695a3d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:07.729495 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7d16b44a-38af-44b9-89a4-68b354695a3d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:07.729918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7d16b44a-38af-44b9-89a4-68b354695a3d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:07.861204 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7d16b44a-38af-44b9-89a4-68b354695a3d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:07.869009 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7d16b44a-38af-44b9-89a4-68b354695a3d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152281 Jun 10 13:14:07.876892 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 566/1100] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:07 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:08.538143 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6577a007-07f6-458d-8581-c136e57e9313 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:08.539076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6577a007-07f6-458d-8581-c136e57e9313 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:08.545459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6577a007-07f6-458d-8581-c136e57e9313 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:08.545899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6577a007-07f6-458d-8581-c136e57e9313 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:08.546658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6577a007-07f6-458d-8581-c136e57e9313 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:08.547076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6577a007-07f6-458d-8581-c136e57e9313 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:08.577171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6577a007-07f6-458d-8581-c136e57e9313 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042905 Jun 10 13:14:08.577717 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 535/1101] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:08 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:08.885933 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3a5d8315-5a00-40a3-be9b-b3b1018694c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:08.886592 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3a5d8315-5a00-40a3-be9b-b3b1018694c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:08.893021 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3a5d8315-5a00-40a3-be9b-b3b1018694c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:08.893469 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3a5d8315-5a00-40a3-be9b-b3b1018694c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:08.894225 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3a5d8315-5a00-40a3-be9b-b3b1018694c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:08.894655 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3a5d8315-5a00-40a3-be9b-b3b1018694c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:09.031432 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3a5d8315-5a00-40a3-be9b-b3b1018694c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:09.038995 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3a5d8315-5a00-40a3-be9b-b3b1018694c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.156929 Jun 10 13:14:09.039564 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 567/1102] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:08 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:09.589746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dbc64ac5-c0b7-4462-b803-a3aa0753a4f1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:09.590600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dbc64ac5-c0b7-4462-b803-a3aa0753a4f1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:09.597060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dbc64ac5-c0b7-4462-b803-a3aa0753a4f1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:09.597501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dbc64ac5-c0b7-4462-b803-a3aa0753a4f1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:09.598257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dbc64ac5-c0b7-4462-b803-a3aa0753a4f1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:09.598695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dbc64ac5-c0b7-4462-b803-a3aa0753a4f1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:09.627026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dbc64ac5-c0b7-4462-b803-a3aa0753a4f1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041453 Jun 10 13:14:09.627580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 536/1103] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:09 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:10.055550 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-192297f8-1c02-4d62-96be-815d2966ce6f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:10.056163 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-192297f8-1c02-4d62-96be-815d2966ce6f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:10.062205 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-192297f8-1c02-4d62-96be-815d2966ce6f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:10.062676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-192297f8-1c02-4d62-96be-815d2966ce6f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:10.063431 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-192297f8-1c02-4d62-96be-815d2966ce6f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:10.063861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-192297f8-1c02-4d62-96be-815d2966ce6f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:10.193026 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-192297f8-1c02-4d62-96be-815d2966ce6f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:10.200433 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-192297f8-1c02-4d62-96be-815d2966ce6f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148922 Jun 10 13:14:10.201016 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 568/1104] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:10 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:10.639597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c57f041d-a064-465d-8d36-872f284bb0fc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:10.640757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c57f041d-a064-465d-8d36-872f284bb0fc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:10.647836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c57f041d-a064-465d-8d36-872f284bb0fc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:10.648284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c57f041d-a064-465d-8d36-872f284bb0fc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:10.649075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c57f041d-a064-465d-8d36-872f284bb0fc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:10.649509 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c57f041d-a064-465d-8d36-872f284bb0fc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:10.664896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:10.680371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c57f041d-a064-465d-8d36-872f284bb0fc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044765 Jun 10 13:14:10.680937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 537/1105] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:10 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:11.217835 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c00075f6-a3d4-4168-a9bd-6da8ff0559a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:11.218483 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c00075f6-a3d4-4168-a9bd-6da8ff0559a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:11.225552 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c00075f6-a3d4-4168-a9bd-6da8ff0559a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:11.226002 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c00075f6-a3d4-4168-a9bd-6da8ff0559a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:11.226771 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c00075f6-a3d4-4168-a9bd-6da8ff0559a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:11.227205 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c00075f6-a3d4-4168-a9bd-6da8ff0559a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:11.358570 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c00075f6-a3d4-4168-a9bd-6da8ff0559a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:11.366805 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c00075f6-a3d4-4168-a9bd-6da8ff0559a4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152816 Jun 10 13:14:11.367371 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 569/1106] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:11 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:11.691981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2db9dbe9-1a3d-4d20-82ce-0c751316d310 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:11.693558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2db9dbe9-1a3d-4d20-82ce-0c751316d310 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:11.699975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2db9dbe9-1a3d-4d20-82ce-0c751316d310 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:11.700414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2db9dbe9-1a3d-4d20-82ce-0c751316d310 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:11.701194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2db9dbe9-1a3d-4d20-82ce-0c751316d310 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:11.701623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2db9dbe9-1a3d-4d20-82ce-0c751316d310 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:11.729015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2db9dbe9-1a3d-4d20-82ce-0c751316d310 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041389 Jun 10 13:14:11.729547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 538/1107] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:11 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:12.384219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b7879139-27c9-451a-8cf2-b2c15fd4f55e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:12.384845 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b7879139-27c9-451a-8cf2-b2c15fd4f55e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:12.391257 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b7879139-27c9-451a-8cf2-b2c15fd4f55e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:12.391698 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b7879139-27c9-451a-8cf2-b2c15fd4f55e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:12.392444 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b7879139-27c9-451a-8cf2-b2c15fd4f55e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:12.392896 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b7879139-27c9-451a-8cf2-b2c15fd4f55e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:12.521438 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b7879139-27c9-451a-8cf2-b2c15fd4f55e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:12.528705 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b7879139-27c9-451a-8cf2-b2c15fd4f55e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148916 Jun 10 13:14:12.529282 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 570/1108] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:12 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:12.740356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-60dab24d-bd35-4542-bc83-90ab2a1b6933 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:12.741308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-60dab24d-bd35-4542-bc83-90ab2a1b6933 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:12.747827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60dab24d-bd35-4542-bc83-90ab2a1b6933 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:12.748265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60dab24d-bd35-4542-bc83-90ab2a1b6933 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:12.749041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60dab24d-bd35-4542-bc83-90ab2a1b6933 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:12.749471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60dab24d-bd35-4542-bc83-90ab2a1b6933 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:12.776817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-60dab24d-bd35-4542-bc83-90ab2a1b6933 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041242 Jun 10 13:14:12.777347 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 539/1109] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:12 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:13.545105 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6e53b388-add5-4f18-b0cf-bedadc14796e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:13.545750 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6e53b388-add5-4f18-b0cf-bedadc14796e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:13.552237 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e53b388-add5-4f18-b0cf-bedadc14796e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:13.552690 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e53b388-add5-4f18-b0cf-bedadc14796e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:13.553483 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e53b388-add5-4f18-b0cf-bedadc14796e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:13.553921 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e53b388-add5-4f18-b0cf-bedadc14796e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:13.680106 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6e53b388-add5-4f18-b0cf-bedadc14796e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:13.687605 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6e53b388-add5-4f18-b0cf-bedadc14796e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146360 Jun 10 13:14:13.688191 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 571/1110] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:13 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:13.788355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d5da62fa-7129-40e1-b614-1ff68a678c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:13.789297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d5da62fa-7129-40e1-b614-1ff68a678c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:13.795655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5da62fa-7129-40e1-b614-1ff68a678c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:13.796100 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5da62fa-7129-40e1-b614-1ff68a678c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:13.796907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5da62fa-7129-40e1-b614-1ff68a678c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:13.797328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5da62fa-7129-40e1-b614-1ff68a678c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:13.825916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d5da62fa-7129-40e1-b614-1ff68a678c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041533 Jun 10 13:14:13.826468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 540/1111] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:13 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:14.704257 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b966e0a6-a003-448a-b8a7-4d5ee59b0649 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:14.704889 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b966e0a6-a003-448a-b8a7-4d5ee59b0649 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:14.711461 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b966e0a6-a003-448a-b8a7-4d5ee59b0649 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:14.711905 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b966e0a6-a003-448a-b8a7-4d5ee59b0649 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:14.712649 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b966e0a6-a003-448a-b8a7-4d5ee59b0649 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:14.713109 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b966e0a6-a003-448a-b8a7-4d5ee59b0649 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:14.837750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b4639302-1403-4ba9-ac85-71dad8cb299d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:14.838658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b4639302-1403-4ba9-ac85-71dad8cb299d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:14.843647 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b966e0a6-a003-448a-b8a7-4d5ee59b0649 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:14.845092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b4639302-1403-4ba9-ac85-71dad8cb299d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:14.845545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b4639302-1403-4ba9-ac85-71dad8cb299d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:14.846314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b4639302-1403-4ba9-ac85-71dad8cb299d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:14.846743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b4639302-1403-4ba9-ac85-71dad8cb299d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:14.851458 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b966e0a6-a003-448a-b8a7-4d5ee59b0649 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151025 Jun 10 13:14:14.852023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 572/1112] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:14 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:14.876492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b4639302-1403-4ba9-ac85-71dad8cb299d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042803 Jun 10 13:14:14.877082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 541/1113] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:14 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:14:15.868590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-597a8e22-01e2-4c14-9e29-38359a8ea1d6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:15.869153 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-597a8e22-01e2-4c14-9e29-38359a8ea1d6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:15.877431 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-597a8e22-01e2-4c14-9e29-38359a8ea1d6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:15.878164 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-597a8e22-01e2-4c14-9e29-38359a8ea1d6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:15.879236 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-597a8e22-01e2-4c14-9e29-38359a8ea1d6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:15.879671 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-597a8e22-01e2-4c14-9e29-38359a8ea1d6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:15.888997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ba5a1230-373b-4099-a3e3-6fbfe4f8778a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:15.889924 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ba5a1230-373b-4099-a3e3-6fbfe4f8778a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:15.896795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba5a1230-373b-4099-a3e3-6fbfe4f8778a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:15.897246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba5a1230-373b-4099-a3e3-6fbfe4f8778a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:15.898020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba5a1230-373b-4099-a3e3-6fbfe4f8778a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:15.898468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba5a1230-373b-4099-a3e3-6fbfe4f8778a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:15.927171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ba5a1230-373b-4099-a3e3-6fbfe4f8778a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042157 Jun 10 13:14:15.927730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 542/1114] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:15 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:14:16.011783 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-597a8e22-01e2-4c14-9e29-38359a8ea1d6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:16.019329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-597a8e22-01e2-4c14-9e29-38359a8ea1d6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.154693 Jun 10 13:14:16.019883 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 573/1115] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:15 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:16.939098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5679e712-e7a9-4054-8408-00a0e17a3029 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:16.940048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5679e712-e7a9-4054-8408-00a0e17a3029 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:16.946561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5679e712-e7a9-4054-8408-00a0e17a3029 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:16.947020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5679e712-e7a9-4054-8408-00a0e17a3029 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:16.947810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5679e712-e7a9-4054-8408-00a0e17a3029 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:16.948246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5679e712-e7a9-4054-8408-00a0e17a3029 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:16.978261 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5679e712-e7a9-4054-8408-00a0e17a3029 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043122 Jun 10 13:14:16.978746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 543/1116] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:16 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:17.037277 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-fc107c5b-8f61-463b-9596-f48f710edaf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:17.037884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-fc107c5b-8f61-463b-9596-f48f710edaf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:17.044250 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fc107c5b-8f61-463b-9596-f48f710edaf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:17.044691 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fc107c5b-8f61-463b-9596-f48f710edaf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:17.045468 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fc107c5b-8f61-463b-9596-f48f710edaf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:17.045897 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fc107c5b-8f61-463b-9596-f48f710edaf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:17.182217 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-fc107c5b-8f61-463b-9596-f48f710edaf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:17.191080 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-fc107c5b-8f61-463b-9596-f48f710edaf3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.158138 Jun 10 13:14:17.193844 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 574/1117] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:17 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:17.990684 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-72f854b9-2337-48ee-877d-dc011c5a1bb3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:17.991591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-72f854b9-2337-48ee-877d-dc011c5a1bb3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:17.997964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72f854b9-2337-48ee-877d-dc011c5a1bb3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:17.998408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72f854b9-2337-48ee-877d-dc011c5a1bb3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:17.999155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72f854b9-2337-48ee-877d-dc011c5a1bb3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:17.999583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72f854b9-2337-48ee-877d-dc011c5a1bb3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:18.027230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-72f854b9-2337-48ee-877d-dc011c5a1bb3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041387 Jun 10 13:14:18.027762 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 544/1118] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:17 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:18.210110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-40884226-793e-4c62-8c8f-d9b64d90a5f1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:18.210736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-40884226-793e-4c62-8c8f-d9b64d90a5f1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:18.217550 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-40884226-793e-4c62-8c8f-d9b64d90a5f1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:18.217994 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-40884226-793e-4c62-8c8f-d9b64d90a5f1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:18.218745 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-40884226-793e-4c62-8c8f-d9b64d90a5f1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:18.219170 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-40884226-793e-4c62-8c8f-d9b64d90a5f1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:18.348822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-40884226-793e-4c62-8c8f-d9b64d90a5f1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:18.354870 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-40884226-793e-4c62-8c8f-d9b64d90a5f1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148684 Jun 10 13:14:18.355428 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 575/1119] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:18 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:19.038108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9d0a1cf6-8436-4684-8c18-7dc430012c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:19.039017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9d0a1cf6-8436-4684-8c18-7dc430012c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:19.045490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9d0a1cf6-8436-4684-8c18-7dc430012c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:19.045934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9d0a1cf6-8436-4684-8c18-7dc430012c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:19.046695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9d0a1cf6-8436-4684-8c18-7dc430012c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:19.047126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9d0a1cf6-8436-4684-8c18-7dc430012c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:19.076535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9d0a1cf6-8436-4684-8c18-7dc430012c77 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042341 Jun 10 13:14:19.077116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 545/1120] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:19 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:19.372098 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0a5f296b-8436-4e86-8994-1e83dbdb370d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:19.372703 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0a5f296b-8436-4e86-8994-1e83dbdb370d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:19.379192 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0a5f296b-8436-4e86-8994-1e83dbdb370d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:19.379631 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0a5f296b-8436-4e86-8994-1e83dbdb370d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:19.381067 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0a5f296b-8436-4e86-8994-1e83dbdb370d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:19.381530 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0a5f296b-8436-4e86-8994-1e83dbdb370d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:19.513882 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0a5f296b-8436-4e86-8994-1e83dbdb370d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:19.523886 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0a5f296b-8436-4e86-8994-1e83dbdb370d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.155608 Jun 10 13:14:19.524500 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 576/1121] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:19 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:20.087903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cb169c7c-7729-4f11-a267-2fedccf204bd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:20.088837 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cb169c7c-7729-4f11-a267-2fedccf204bd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:20.095446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb169c7c-7729-4f11-a267-2fedccf204bd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:20.095912 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb169c7c-7729-4f11-a267-2fedccf204bd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:20.096662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb169c7c-7729-4f11-a267-2fedccf204bd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:20.097139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb169c7c-7729-4f11-a267-2fedccf204bd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:20.125875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cb169c7c-7729-4f11-a267-2fedccf204bd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042492 Jun 10 13:14:20.126420 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 546/1122] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:20 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:14:20.554465 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-65f5b08a-65c6-499d-99f6-bbdec5ea1c9b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:20.555106 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-65f5b08a-65c6-499d-99f6-bbdec5ea1c9b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:20.561662 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-65f5b08a-65c6-499d-99f6-bbdec5ea1c9b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:20.562106 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-65f5b08a-65c6-499d-99f6-bbdec5ea1c9b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:20.562869 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-65f5b08a-65c6-499d-99f6-bbdec5ea1c9b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:20.563361 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-65f5b08a-65c6-499d-99f6-bbdec5ea1c9b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:20.689063 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-65f5b08a-65c6-499d-99f6-bbdec5ea1c9b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:20.697125 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-65f5b08a-65c6-499d-99f6-bbdec5ea1c9b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147131 Jun 10 13:14:20.700138 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 577/1123] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:20 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:14:21.137637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0f4e2f29-1f5f-4504-b0c5-cdb4e0223cae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:21.138540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0f4e2f29-1f5f-4504-b0c5-cdb4e0223cae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:21.144974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f4e2f29-1f5f-4504-b0c5-cdb4e0223cae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:21.145428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f4e2f29-1f5f-4504-b0c5-cdb4e0223cae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:21.146163 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f4e2f29-1f5f-4504-b0c5-cdb4e0223cae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:21.146590 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f4e2f29-1f5f-4504-b0c5-cdb4e0223cae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:21.174535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0f4e2f29-1f5f-4504-b0c5-cdb4e0223cae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040791 Jun 10 13:14:21.175008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 547/1124] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:21 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:21.714994 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0ca43409-e925-4422-bdee-a7f730671a0c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:21.715604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0ca43409-e925-4422-bdee-a7f730671a0c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:21.722110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0ca43409-e925-4422-bdee-a7f730671a0c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:21.722555 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0ca43409-e925-4422-bdee-a7f730671a0c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:21.723314 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0ca43409-e925-4422-bdee-a7f730671a0c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:21.723751 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0ca43409-e925-4422-bdee-a7f730671a0c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:21.741193 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:21.858246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0ca43409-e925-4422-bdee-a7f730671a0c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:21.866011 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0ca43409-e925-4422-bdee-a7f730671a0c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.154883 Jun 10 13:14:21.866576 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 578/1125] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:21 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:14:22.187126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e82bce95-11eb-4a7a-98e6-9dc542b10fa8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:22.187963 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e82bce95-11eb-4a7a-98e6-9dc542b10fa8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:22.190270 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=60 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:22.195020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e82bce95-11eb-4a7a-98e6-9dc542b10fa8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:22.195472 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e82bce95-11eb-4a7a-98e6-9dc542b10fa8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:22.196854 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e82bce95-11eb-4a7a-98e6-9dc542b10fa8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:22.197293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e82bce95-11eb-4a7a-98e6-9dc542b10fa8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:22.225572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e82bce95-11eb-4a7a-98e6-9dc542b10fa8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042477 Jun 10 13:14:22.226112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 548/1126] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:22 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:22.883004 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-065a8506-6554-4093-8654-5b4505b55bf5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:22.883594 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-065a8506-6554-4093-8654-5b4505b55bf5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:22.890038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-065a8506-6554-4093-8654-5b4505b55bf5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:22.890478 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-065a8506-6554-4093-8654-5b4505b55bf5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:22.891231 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-065a8506-6554-4093-8654-5b4505b55bf5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:22.891661 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-065a8506-6554-4093-8654-5b4505b55bf5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:23.019310 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-065a8506-6554-4093-8654-5b4505b55bf5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:23.027482 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-065a8506-6554-4093-8654-5b4505b55bf5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148336 Jun 10 13:14:23.028038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 579/1127] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:22 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:23.237202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ca45f90e-43da-4a7c-85b2-098f7e5bcbe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:23.238119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ca45f90e-43da-4a7c-85b2-098f7e5bcbe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:23.244722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca45f90e-43da-4a7c-85b2-098f7e5bcbe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:23.245210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca45f90e-43da-4a7c-85b2-098f7e5bcbe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:23.245972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca45f90e-43da-4a7c-85b2-098f7e5bcbe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:23.246516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca45f90e-43da-4a7c-85b2-098f7e5bcbe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:23.290126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ca45f90e-43da-4a7c-85b2-098f7e5bcbe5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057403 Jun 10 13:14:23.290683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 549/1128] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:23 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:24.045068 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3d0dd0a6-1ef8-4065-bcb7-7dd6f0cc7165 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:24.045678 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3d0dd0a6-1ef8-4065-bcb7-7dd6f0cc7165 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:24.052070 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d0dd0a6-1ef8-4065-bcb7-7dd6f0cc7165 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:24.052518 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d0dd0a6-1ef8-4065-bcb7-7dd6f0cc7165 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:24.053325 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d0dd0a6-1ef8-4065-bcb7-7dd6f0cc7165 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:24.053760 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3d0dd0a6-1ef8-4065-bcb7-7dd6f0cc7165 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:24.179875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3d0dd0a6-1ef8-4065-bcb7-7dd6f0cc7165 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:24.187991 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3d0dd0a6-1ef8-4065-bcb7-7dd6f0cc7165 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147371 Jun 10 13:14:24.188550 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 580/1129] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:24 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:24.302452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-96d02e72-7c8a-4433-8ece-a1888e9cd7fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:24.303913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-96d02e72-7c8a-4433-8ece-a1888e9cd7fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:24.310850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96d02e72-7c8a-4433-8ece-a1888e9cd7fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:24.311653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96d02e72-7c8a-4433-8ece-a1888e9cd7fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:24.312735 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96d02e72-7c8a-4433-8ece-a1888e9cd7fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:24.313203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96d02e72-7c8a-4433-8ece-a1888e9cd7fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:24.343649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-96d02e72-7c8a-4433-8ece-a1888e9cd7fa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045485 Jun 10 13:14:24.344198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 550/1130] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:24 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:25.205446 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-525a6cb7-bdf1-4af6-9bb1-0feeedd5bb7e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:25.206061 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-525a6cb7-bdf1-4af6-9bb1-0feeedd5bb7e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:25.212734 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-525a6cb7-bdf1-4af6-9bb1-0feeedd5bb7e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:25.213285 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-525a6cb7-bdf1-4af6-9bb1-0feeedd5bb7e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:25.214042 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-525a6cb7-bdf1-4af6-9bb1-0feeedd5bb7e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:25.214469 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-525a6cb7-bdf1-4af6-9bb1-0feeedd5bb7e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:25.344127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-525a6cb7-bdf1-4af6-9bb1-0feeedd5bb7e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:25.351523 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-525a6cb7-bdf1-4af6-9bb1-0feeedd5bb7e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149906 Jun 10 13:14:25.352077 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 581/1131] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:25 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:25.355804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-139ca4ce-b6ac-485c-9a12-34a6ae7e5867 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:25.356720 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-139ca4ce-b6ac-485c-9a12-34a6ae7e5867 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:25.363336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-139ca4ce-b6ac-485c-9a12-34a6ae7e5867 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:25.363782 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-139ca4ce-b6ac-485c-9a12-34a6ae7e5867 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:25.364542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-139ca4ce-b6ac-485c-9a12-34a6ae7e5867 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:25.365004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-139ca4ce-b6ac-485c-9a12-34a6ae7e5867 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:25.393112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-139ca4ce-b6ac-485c-9a12-34a6ae7e5867 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041827 Jun 10 13:14:25.393660 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 551/1132] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:25 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:14:26.368217 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-797ceed5-b875-4570-a80a-c6348521d8e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:26.368848 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-797ceed5-b875-4570-a80a-c6348521d8e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:26.375359 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-797ceed5-b875-4570-a80a-c6348521d8e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:26.375798 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-797ceed5-b875-4570-a80a-c6348521d8e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:26.376542 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-797ceed5-b875-4570-a80a-c6348521d8e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:26.376992 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-797ceed5-b875-4570-a80a-c6348521d8e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:26.404466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dca716a1-faeb-4061-884c-1661aea3b272 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:26.405434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dca716a1-faeb-4061-884c-1661aea3b272 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:26.411905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dca716a1-faeb-4061-884c-1661aea3b272 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:26.412352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dca716a1-faeb-4061-884c-1661aea3b272 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:26.413138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dca716a1-faeb-4061-884c-1661aea3b272 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:26.413572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dca716a1-faeb-4061-884c-1661aea3b272 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:26.442412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dca716a1-faeb-4061-884c-1661aea3b272 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042502 Jun 10 13:14:26.442961 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 552/1133] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:26 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:14:26.504001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-797ceed5-b875-4570-a80a-c6348521d8e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:26.512114 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-797ceed5-b875-4570-a80a-c6348521d8e1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147811 Jun 10 13:14:26.512674 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 582/1134] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:26 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:27.454068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bdd69508-9e92-48fc-aed2-fbcd7e5b6dc8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:27.454970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bdd69508-9e92-48fc-aed2-fbcd7e5b6dc8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:27.461498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bdd69508-9e92-48fc-aed2-fbcd7e5b6dc8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:27.461942 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bdd69508-9e92-48fc-aed2-fbcd7e5b6dc8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:27.463213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bdd69508-9e92-48fc-aed2-fbcd7e5b6dc8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:27.463644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bdd69508-9e92-48fc-aed2-fbcd7e5b6dc8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:27.493859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bdd69508-9e92-48fc-aed2-fbcd7e5b6dc8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043863 Jun 10 13:14:27.494393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 553/1135] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:27 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:27.529632 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0f0e2640-ac89-4dc5-a462-393b4769c20a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:27.530231 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0f0e2640-ac89-4dc5-a462-393b4769c20a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:27.536587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f0e2640-ac89-4dc5-a462-393b4769c20a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:27.537094 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f0e2640-ac89-4dc5-a462-393b4769c20a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:27.537844 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f0e2640-ac89-4dc5-a462-393b4769c20a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:27.538269 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f0e2640-ac89-4dc5-a462-393b4769c20a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:27.670613 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0f0e2640-ac89-4dc5-a462-393b4769c20a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:27.678137 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0f0e2640-ac89-4dc5-a462-393b4769c20a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152822 Jun 10 13:14:27.678698 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 583/1136] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:27 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:28.505481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1d8088b0-ade2-4e0b-9aa7-7263b5ada6c9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:28.506410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1d8088b0-ade2-4e0b-9aa7-7263b5ada6c9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:28.512836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d8088b0-ade2-4e0b-9aa7-7263b5ada6c9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:28.513286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d8088b0-ade2-4e0b-9aa7-7263b5ada6c9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:28.514042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d8088b0-ade2-4e0b-9aa7-7263b5ada6c9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:28.514474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d8088b0-ade2-4e0b-9aa7-7263b5ada6c9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:28.516802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:28.543433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1d8088b0-ade2-4e0b-9aa7-7263b5ada6c9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041967 Jun 10 13:14:28.543974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 554/1137] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:28 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:28.694931 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-fb9ee6e8-f4ae-4d95-ae6c-c542b86dc0c4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:28.695539 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-fb9ee6e8-f4ae-4d95-ae6c-c542b86dc0c4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:28.702405 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fb9ee6e8-f4ae-4d95-ae6c-c542b86dc0c4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:28.702877 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fb9ee6e8-f4ae-4d95-ae6c-c542b86dc0c4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:28.703644 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fb9ee6e8-f4ae-4d95-ae6c-c542b86dc0c4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:28.704074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fb9ee6e8-f4ae-4d95-ae6c-c542b86dc0c4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:28.830253 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-fb9ee6e8-f4ae-4d95-ae6c-c542b86dc0c4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:28.838366 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-fb9ee6e8-f4ae-4d95-ae6c-c542b86dc0c4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146851 Jun 10 13:14:28.838932 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 584/1138] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:28 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:29.555372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bab4da55-876e-4ed1-933b-6e3725aa627a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:29.556269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bab4da55-876e-4ed1-933b-6e3725aa627a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:29.563286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bab4da55-876e-4ed1-933b-6e3725aa627a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:29.563728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bab4da55-876e-4ed1-933b-6e3725aa627a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:29.564476 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bab4da55-876e-4ed1-933b-6e3725aa627a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:29.564943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bab4da55-876e-4ed1-933b-6e3725aa627a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:29.593869 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bab4da55-876e-4ed1-933b-6e3725aa627a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042784 Jun 10 13:14:29.594357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 555/1139] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:29 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:29.855577 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-113565b7-4e80-475a-a247-b3890ac28c59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:29.856109 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-113565b7-4e80-475a-a247-b3890ac28c59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:29.862902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-113565b7-4e80-475a-a247-b3890ac28c59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:29.863350 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-113565b7-4e80-475a-a247-b3890ac28c59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:29.864109 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-113565b7-4e80-475a-a247-b3890ac28c59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:29.864538 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-113565b7-4e80-475a-a247-b3890ac28c59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:29.992886 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-113565b7-4e80-475a-a247-b3890ac28c59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:30.000979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-113565b7-4e80-475a-a247-b3890ac28c59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149847 Jun 10 13:14:30.001542 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 585/1140] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:29 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:30.605560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-602459bc-5e6c-4231-866c-6100649be584 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:30.606393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-602459bc-5e6c-4231-866c-6100649be584 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:30.618885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-602459bc-5e6c-4231-866c-6100649be584 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:30.619326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-602459bc-5e6c-4231-866c-6100649be584 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:30.620086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-602459bc-5e6c-4231-866c-6100649be584 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:30.620514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-602459bc-5e6c-4231-866c-6100649be584 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:30.648629 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-602459bc-5e6c-4231-866c-6100649be584 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047151 Jun 10 13:14:30.649201 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 556/1141] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:30 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:31.017860 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-97803b12-7c77-4e7b-8dc1-838321af7ea3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:31.018466 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-97803b12-7c77-4e7b-8dc1-838321af7ea3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:31.024959 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-97803b12-7c77-4e7b-8dc1-838321af7ea3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:31.025397 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-97803b12-7c77-4e7b-8dc1-838321af7ea3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:31.026146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-97803b12-7c77-4e7b-8dc1-838321af7ea3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:31.026570 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-97803b12-7c77-4e7b-8dc1-838321af7ea3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:31.159943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-97803b12-7c77-4e7b-8dc1-838321af7ea3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:31.167332 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-97803b12-7c77-4e7b-8dc1-838321af7ea3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.153354 Jun 10 13:14:31.167921 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 586/1142] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:31 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:31.661075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-139c0de0-92ae-4229-9ee1-4198e5a4257d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:31.662018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-139c0de0-92ae-4229-9ee1-4198e5a4257d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:31.668644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-139c0de0-92ae-4229-9ee1-4198e5a4257d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:31.669118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-139c0de0-92ae-4229-9ee1-4198e5a4257d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:31.669886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-139c0de0-92ae-4229-9ee1-4198e5a4257d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:31.670321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-139c0de0-92ae-4229-9ee1-4198e5a4257d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:31.698173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-139c0de0-92ae-4229-9ee1-4198e5a4257d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041257 Jun 10 13:14:31.698713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 557/1143] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:31 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:32.183241 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e000b3cf-6570-40ca-91f1-5db8ff5e7a19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:32.184231 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e000b3cf-6570-40ca-91f1-5db8ff5e7a19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:32.186674 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=52 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:32.191355 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e000b3cf-6570-40ca-91f1-5db8ff5e7a19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:32.191796 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e000b3cf-6570-40ca-91f1-5db8ff5e7a19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:32.192540 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e000b3cf-6570-40ca-91f1-5db8ff5e7a19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:32.193037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e000b3cf-6570-40ca-91f1-5db8ff5e7a19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:32.320194 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e000b3cf-6570-40ca-91f1-5db8ff5e7a19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:32.328343 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e000b3cf-6570-40ca-91f1-5db8ff5e7a19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148916 Jun 10 13:14:32.328935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 587/1144] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:32 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:32.710096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9390f75a-a859-4c45-b252-55c799273960 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:32.711007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9390f75a-a859-4c45-b252-55c799273960 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:32.717485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9390f75a-a859-4c45-b252-55c799273960 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:32.717933 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9390f75a-a859-4c45-b252-55c799273960 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:32.718683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9390f75a-a859-4c45-b252-55c799273960 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:32.719114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9390f75a-a859-4c45-b252-55c799273960 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:32.748082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9390f75a-a859-4c45-b252-55c799273960 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041938 Jun 10 13:14:32.748631 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 558/1145] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:32 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:33.345583 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c9eafa76-b934-48d1-b242-d7eebda6317c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:33.346186 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c9eafa76-b934-48d1-b242-d7eebda6317c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:33.352595 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c9eafa76-b934-48d1-b242-d7eebda6317c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:33.353068 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c9eafa76-b934-48d1-b242-d7eebda6317c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:33.353817 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c9eafa76-b934-48d1-b242-d7eebda6317c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:33.354911 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c9eafa76-b934-48d1-b242-d7eebda6317c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:33.482741 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c9eafa76-b934-48d1-b242-d7eebda6317c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:33.490449 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c9eafa76-b934-48d1-b242-d7eebda6317c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148770 Jun 10 13:14:33.491004 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 588/1146] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:33 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:14:33.759643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-33299c00-316d-47a2-894b-60388fb9e4f3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:33.760600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-33299c00-316d-47a2-894b-60388fb9e4f3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:33.767716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33299c00-316d-47a2-894b-60388fb9e4f3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:33.768161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33299c00-316d-47a2-894b-60388fb9e4f3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:33.768955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33299c00-316d-47a2-894b-60388fb9e4f3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:33.769393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33299c00-316d-47a2-894b-60388fb9e4f3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:33.797841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-33299c00-316d-47a2-894b-60388fb9e4f3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042291 Jun 10 13:14:33.798373 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 559/1147] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:33 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:34.507280 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c0e06a6e-b432-4118-a544-ab90f07206c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:34.507891 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c0e06a6e-b432-4118-a544-ab90f07206c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:34.514911 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c0e06a6e-b432-4118-a544-ab90f07206c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:34.515391 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c0e06a6e-b432-4118-a544-ab90f07206c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:34.516154 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c0e06a6e-b432-4118-a544-ab90f07206c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:34.516592 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c0e06a6e-b432-4118-a544-ab90f07206c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:34.645545 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c0e06a6e-b432-4118-a544-ab90f07206c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:34.653458 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c0e06a6e-b432-4118-a544-ab90f07206c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150058 Jun 10 13:14:34.654013 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 589/1148] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:34 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:34.809368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-554b22c9-dd0b-435b-827e-a250f0448581 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:34.810284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-554b22c9-dd0b-435b-827e-a250f0448581 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:34.816722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-554b22c9-dd0b-435b-827e-a250f0448581 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:34.817203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-554b22c9-dd0b-435b-827e-a250f0448581 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:34.817965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-554b22c9-dd0b-435b-827e-a250f0448581 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:34.818403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-554b22c9-dd0b-435b-827e-a250f0448581 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:34.846916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-554b22c9-dd0b-435b-827e-a250f0448581 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041464 Jun 10 13:14:34.847400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 560/1149] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:34 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:35.670939 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a6cccecf-59de-4dcc-b2dc-d650dc66e204 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:35.671471 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a6cccecf-59de-4dcc-b2dc-d650dc66e204 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:35.677960 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a6cccecf-59de-4dcc-b2dc-d650dc66e204 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:35.678404 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a6cccecf-59de-4dcc-b2dc-d650dc66e204 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:35.679152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a6cccecf-59de-4dcc-b2dc-d650dc66e204 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:35.679581 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a6cccecf-59de-4dcc-b2dc-d650dc66e204 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:35.808849 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a6cccecf-59de-4dcc-b2dc-d650dc66e204 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:35.816284 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a6cccecf-59de-4dcc-b2dc-d650dc66e204 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149887 Jun 10 13:14:35.816863 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 590/1150] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:35 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:35.858020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-106fbb90-b3e0-44bc-8ba0-a1715690c4a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:35.858973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-106fbb90-b3e0-44bc-8ba0-a1715690c4a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:35.865656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-106fbb90-b3e0-44bc-8ba0-a1715690c4a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:35.866108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-106fbb90-b3e0-44bc-8ba0-a1715690c4a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:35.866870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-106fbb90-b3e0-44bc-8ba0-a1715690c4a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:35.867310 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-106fbb90-b3e0-44bc-8ba0-a1715690c4a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:35.895519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-106fbb90-b3e0-44bc-8ba0-a1715690c4a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041450 Jun 10 13:14:35.896053 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 561/1151] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:35 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:36.834526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-bb4fcce7-e6f1-4cda-bdf7-b669ed0915e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:36.835133 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-bb4fcce7-e6f1-4cda-bdf7-b669ed0915e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:36.841705 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bb4fcce7-e6f1-4cda-bdf7-b669ed0915e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:36.842150 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bb4fcce7-e6f1-4cda-bdf7-b669ed0915e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:36.842906 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bb4fcce7-e6f1-4cda-bdf7-b669ed0915e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:36.843339 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-bb4fcce7-e6f1-4cda-bdf7-b669ed0915e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:36.907758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-16fbefb3-af65-4885-b8ff-36d0406b7aa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:36.908787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-16fbefb3-af65-4885-b8ff-36d0406b7aa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:36.915361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-16fbefb3-af65-4885-b8ff-36d0406b7aa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:36.921064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-16fbefb3-af65-4885-b8ff-36d0406b7aa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:36.921064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-16fbefb3-af65-4885-b8ff-36d0406b7aa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:36.921064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-16fbefb3-af65-4885-b8ff-36d0406b7aa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:36.949157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-16fbefb3-af65-4885-b8ff-36d0406b7aa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045421 Jun 10 13:14:36.949764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 562/1152] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:36 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:14:36.977278 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-bb4fcce7-e6f1-4cda-bdf7-b669ed0915e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:36.985289 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-bb4fcce7-e6f1-4cda-bdf7-b669ed0915e5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.154657 Jun 10 13:14:36.985855 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 591/1153] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:36 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:37.960021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-32e83105-7f02-4f20-9584-e3f9a2476570 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:37.960967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-32e83105-7f02-4f20-9584-e3f9a2476570 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:37.967847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-32e83105-7f02-4f20-9584-e3f9a2476570 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:37.968282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-32e83105-7f02-4f20-9584-e3f9a2476570 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:37.969056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-32e83105-7f02-4f20-9584-e3f9a2476570 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:37.969484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-32e83105-7f02-4f20-9584-e3f9a2476570 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:37.998378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-32e83105-7f02-4f20-9584-e3f9a2476570 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042286 Jun 10 13:14:37.998915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 563/1154] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:37 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:38.002362 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-668430fc-9c84-4fe5-a3db-a86b067dd6eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:38.002968 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-668430fc-9c84-4fe5-a3db-a86b067dd6eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:38.009446 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-668430fc-9c84-4fe5-a3db-a86b067dd6eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:38.009889 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-668430fc-9c84-4fe5-a3db-a86b067dd6eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:38.010642 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-668430fc-9c84-4fe5-a3db-a86b067dd6eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:38.011074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-668430fc-9c84-4fe5-a3db-a86b067dd6eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:38.135759 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-668430fc-9c84-4fe5-a3db-a86b067dd6eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:38.143621 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-668430fc-9c84-4fe5-a3db-a86b067dd6eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145047 Jun 10 13:14:38.144177 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 592/1155] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:37 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:14:39.009818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3a180917-2c0b-496c-a5dc-90e97954260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:39.010770 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3a180917-2c0b-496c-a5dc-90e97954260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:39.017801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3a180917-2c0b-496c-a5dc-90e97954260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:39.018257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3a180917-2c0b-496c-a5dc-90e97954260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:39.019014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3a180917-2c0b-496c-a5dc-90e97954260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:39.019477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3a180917-2c0b-496c-a5dc-90e97954260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:39.048080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3a180917-2c0b-496c-a5dc-90e97954260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042327 Jun 10 13:14:39.048624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 564/1156] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:39 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:39.160503 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3873dbd9-e056-408a-b277-6c05f95ae079 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:39.161134 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3873dbd9-e056-408a-b277-6c05f95ae079 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:39.167164 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3873dbd9-e056-408a-b277-6c05f95ae079 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:39.167606 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3873dbd9-e056-408a-b277-6c05f95ae079 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:39.168359 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3873dbd9-e056-408a-b277-6c05f95ae079 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:39.168808 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3873dbd9-e056-408a-b277-6c05f95ae079 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:39.298234 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3873dbd9-e056-408a-b277-6c05f95ae079 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:39.305663 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3873dbd9-e056-408a-b277-6c05f95ae079 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149107 Jun 10 13:14:39.306226 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 593/1157] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:39 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:14:40.059516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cd47115f-69e6-4648-9743-6bf149b8cb15 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:40.060420 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cd47115f-69e6-4648-9743-6bf149b8cb15 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:40.066883 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cd47115f-69e6-4648-9743-6bf149b8cb15 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:40.067346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cd47115f-69e6-4648-9743-6bf149b8cb15 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:40.068091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cd47115f-69e6-4648-9743-6bf149b8cb15 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:40.068515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cd47115f-69e6-4648-9743-6bf149b8cb15 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:40.096947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cd47115f-69e6-4648-9743-6bf149b8cb15 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041344 Jun 10 13:14:40.097492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 565/1158] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:40 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:40.323332 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f9003a0b-cc72-4875-a6f3-645e5392fee7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:40.323842 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f9003a0b-cc72-4875-a6f3-645e5392fee7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:40.330981 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f9003a0b-cc72-4875-a6f3-645e5392fee7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:40.331426 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f9003a0b-cc72-4875-a6f3-645e5392fee7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:40.332185 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f9003a0b-cc72-4875-a6f3-645e5392fee7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:40.332615 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f9003a0b-cc72-4875-a6f3-645e5392fee7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:40.458732 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f9003a0b-cc72-4875-a6f3-645e5392fee7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:40.466112 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f9003a0b-cc72-4875-a6f3-645e5392fee7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146758 Jun 10 13:14:40.466680 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 594/1159] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:40 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:41.108507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b9d2e83d-a104-464d-a1ed-df79aa4f85b9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:41.109447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b9d2e83d-a104-464d-a1ed-df79aa4f85b9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:41.116139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9d2e83d-a104-464d-a1ed-df79aa4f85b9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:41.116588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9d2e83d-a104-464d-a1ed-df79aa4f85b9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:41.117398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9d2e83d-a104-464d-a1ed-df79aa4f85b9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:41.117840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9d2e83d-a104-464d-a1ed-df79aa4f85b9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:41.146161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b9d2e83d-a104-464d-a1ed-df79aa4f85b9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041720 Jun 10 13:14:41.146706 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 566/1160] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:41 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:41.483122 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9c57dc82-d82b-44cc-bf66-9af64872eb6c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:41.483755 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9c57dc82-d82b-44cc-bf66-9af64872eb6c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:41.490560 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9c57dc82-d82b-44cc-bf66-9af64872eb6c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:41.491004 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9c57dc82-d82b-44cc-bf66-9af64872eb6c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:41.491759 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9c57dc82-d82b-44cc-bf66-9af64872eb6c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:41.492188 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9c57dc82-d82b-44cc-bf66-9af64872eb6c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:41.496133 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:41.622203 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9c57dc82-d82b-44cc-bf66-9af64872eb6c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:41.630353 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9c57dc82-d82b-44cc-bf66-9af64872eb6c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151710 Jun 10 13:14:41.630921 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 595/1161] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:41 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:42.157727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d396dfa2-b57c-4b6b-a72b-4c8169e9fd85 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:42.158668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d396dfa2-b57c-4b6b-a72b-4c8169e9fd85 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:42.165129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d396dfa2-b57c-4b6b-a72b-4c8169e9fd85 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:42.165581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d396dfa2-b57c-4b6b-a72b-4c8169e9fd85 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:42.166333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d396dfa2-b57c-4b6b-a72b-4c8169e9fd85 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:42.166771 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d396dfa2-b57c-4b6b-a72b-4c8169e9fd85 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:42.195286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d396dfa2-b57c-4b6b-a72b-4c8169e9fd85 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041596 Jun 10 13:14:42.195829 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 567/1162] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:42 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:42.647857 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2db6e4d7-69cb-4d37-af20-700160f4c812 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:42.648402 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2db6e4d7-69cb-4d37-af20-700160f4c812 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:42.655078 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2db6e4d7-69cb-4d37-af20-700160f4c812 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:42.655521 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2db6e4d7-69cb-4d37-af20-700160f4c812 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:42.656277 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2db6e4d7-69cb-4d37-af20-700160f4c812 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:42.656707 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2db6e4d7-69cb-4d37-af20-700160f4c812 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:42.784286 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2db6e4d7-69cb-4d37-af20-700160f4c812 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:42.791640 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2db6e4d7-69cb-4d37-af20-700160f4c812 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147731 Jun 10 13:14:42.792202 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 596/1163] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:42 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:43.206967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e0889004-c919-4367-962c-033ca15095ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:43.207983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e0889004-c919-4367-962c-033ca15095ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:43.214545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0889004-c919-4367-962c-033ca15095ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:43.214989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0889004-c919-4367-962c-033ca15095ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:43.215748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0889004-c919-4367-962c-033ca15095ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:43.216185 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0889004-c919-4367-962c-033ca15095ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:43.245839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e0889004-c919-4367-962c-033ca15095ef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043014 Jun 10 13:14:43.246388 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 568/1164] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:43 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:43.810251 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d5bfb9fe-4276-42af-bc21-12f0d9af727e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:43.810867 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d5bfb9fe-4276-42af-bc21-12f0d9af727e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:43.817452 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d5bfb9fe-4276-42af-bc21-12f0d9af727e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:43.817901 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d5bfb9fe-4276-42af-bc21-12f0d9af727e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:43.818673 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d5bfb9fe-4276-42af-bc21-12f0d9af727e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:43.819105 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d5bfb9fe-4276-42af-bc21-12f0d9af727e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:43.949263 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d5bfb9fe-4276-42af-bc21-12f0d9af727e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:43.957382 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d5bfb9fe-4276-42af-bc21-12f0d9af727e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152004 Jun 10 13:14:43.957948 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 597/1165] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:43 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:44.258199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2db7bf4b-a0c8-41bc-8fd9-4c901c801385 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:44.259107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2db7bf4b-a0c8-41bc-8fd9-4c901c801385 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:44.266208 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2db7bf4b-a0c8-41bc-8fd9-4c901c801385 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:44.266675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2db7bf4b-a0c8-41bc-8fd9-4c901c801385 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:44.267441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2db7bf4b-a0c8-41bc-8fd9-4c901c801385 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:44.267871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2db7bf4b-a0c8-41bc-8fd9-4c901c801385 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:44.296930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2db7bf4b-a0c8-41bc-8fd9-4c901c801385 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042717 Jun 10 13:14:44.297479 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 569/1166] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:44 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:44.975058 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f449fc51-ac32-4e29-be0b-dd9961e62da2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:44.975691 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f449fc51-ac32-4e29-be0b-dd9961e62da2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:44.982811 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f449fc51-ac32-4e29-be0b-dd9961e62da2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:44.983253 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f449fc51-ac32-4e29-be0b-dd9961e62da2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:44.984004 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f449fc51-ac32-4e29-be0b-dd9961e62da2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:44.984431 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f449fc51-ac32-4e29-be0b-dd9961e62da2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:45.116572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f449fc51-ac32-4e29-be0b-dd9961e62da2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:45.124520 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f449fc51-ac32-4e29-be0b-dd9961e62da2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.153348 Jun 10 13:14:45.125113 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 598/1167] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:44 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:45.308625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c94fa30f-3415-4baa-af02-805690a94776 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:45.310103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c94fa30f-3415-4baa-af02-805690a94776 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:45.316502 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c94fa30f-3415-4baa-af02-805690a94776 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:45.316983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c94fa30f-3415-4baa-af02-805690a94776 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:45.321343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c94fa30f-3415-4baa-af02-805690a94776 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:45.321343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c94fa30f-3415-4baa-af02-805690a94776 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:45.343400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:45.349439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c94fa30f-3415-4baa-af02-805690a94776 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044858 Jun 10 13:14:45.350024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 570/1168] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:45 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:46.141923 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-20330232-e04b-4867-82dd-f6d2e39ffe2b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:46.142564 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-20330232-e04b-4867-82dd-f6d2e39ffe2b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:46.149083 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-20330232-e04b-4867-82dd-f6d2e39ffe2b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:46.149531 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-20330232-e04b-4867-82dd-f6d2e39ffe2b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:46.150290 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-20330232-e04b-4867-82dd-f6d2e39ffe2b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:46.150725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-20330232-e04b-4867-82dd-f6d2e39ffe2b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:46.283706 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-20330232-e04b-4867-82dd-f6d2e39ffe2b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:46.291101 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-20330232-e04b-4867-82dd-f6d2e39ffe2b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.153541 Jun 10 13:14:46.291604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 599/1169] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:46 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:46.361824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8edf3638-3ea6-4382-82ac-72e9651eb651 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:46.362753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8edf3638-3ea6-4382-82ac-72e9651eb651 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:46.369161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8edf3638-3ea6-4382-82ac-72e9651eb651 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:46.369610 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8edf3638-3ea6-4382-82ac-72e9651eb651 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:46.370352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8edf3638-3ea6-4382-82ac-72e9651eb651 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:46.370781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8edf3638-3ea6-4382-82ac-72e9651eb651 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:46.398676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8edf3638-3ea6-4382-82ac-72e9651eb651 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041516 Jun 10 13:14:46.399209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 571/1170] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:46 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:47.308062 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8bffc7d1-03eb-4a04-b213-7248bc241ce2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:47.308675 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8bffc7d1-03eb-4a04-b213-7248bc241ce2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:47.315302 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8bffc7d1-03eb-4a04-b213-7248bc241ce2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:47.315743 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8bffc7d1-03eb-4a04-b213-7248bc241ce2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:47.316493 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8bffc7d1-03eb-4a04-b213-7248bc241ce2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:47.316954 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8bffc7d1-03eb-4a04-b213-7248bc241ce2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:47.409571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c72b78e7-2cdd-4427-84d6-52d3605a43fd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:47.410700 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c72b78e7-2cdd-4427-84d6-52d3605a43fd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:47.417259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c72b78e7-2cdd-4427-84d6-52d3605a43fd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:47.417711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c72b78e7-2cdd-4427-84d6-52d3605a43fd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:47.418470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c72b78e7-2cdd-4427-84d6-52d3605a43fd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:47.418907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c72b78e7-2cdd-4427-84d6-52d3605a43fd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:47.447495 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c72b78e7-2cdd-4427-84d6-52d3605a43fd tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041976 Jun 10 13:14:47.448062 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 572/1171] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:47 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:14:47.448621 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8bffc7d1-03eb-4a04-b213-7248bc241ce2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:47.456699 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8bffc7d1-03eb-4a04-b213-7248bc241ce2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152567 Jun 10 13:14:47.457273 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 600/1172] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:47 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:48.459711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f6c57a0d-1a14-40b9-b532-5b8b8ef1a593 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:48.460651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f6c57a0d-1a14-40b9-b532-5b8b8ef1a593 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:48.467186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f6c57a0d-1a14-40b9-b532-5b8b8ef1a593 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:48.467632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f6c57a0d-1a14-40b9-b532-5b8b8ef1a593 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:48.468389 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f6c57a0d-1a14-40b9-b532-5b8b8ef1a593 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:48.468841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f6c57a0d-1a14-40b9-b532-5b8b8ef1a593 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:48.473545 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7c185722-6720-4943-876f-f1231ae2d6ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:48.477054 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7c185722-6720-4943-876f-f1231ae2d6ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:48.480732 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7c185722-6720-4943-876f-f1231ae2d6ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:48.481723 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7c185722-6720-4943-876f-f1231ae2d6ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:48.482542 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7c185722-6720-4943-876f-f1231ae2d6ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:48.482979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7c185722-6720-4943-876f-f1231ae2d6ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:48.498931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f6c57a0d-1a14-40b9-b532-5b8b8ef1a593 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043195 Jun 10 13:14:48.499509 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 573/1173] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:48 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:48.608232 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7c185722-6720-4943-876f-f1231ae2d6ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:48.616269 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7c185722-6720-4943-876f-f1231ae2d6ad tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147016 Jun 10 13:14:48.616856 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 601/1174] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:48 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:14:49.510597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2de1f252-b25d-4c2f-96cf-bb9f415d22a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:49.511513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2de1f252-b25d-4c2f-96cf-bb9f415d22a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:49.517939 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2de1f252-b25d-4c2f-96cf-bb9f415d22a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:49.518383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2de1f252-b25d-4c2f-96cf-bb9f415d22a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:49.519134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2de1f252-b25d-4c2f-96cf-bb9f415d22a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:49.519568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2de1f252-b25d-4c2f-96cf-bb9f415d22a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:49.547589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2de1f252-b25d-4c2f-96cf-bb9f415d22a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040998 Jun 10 13:14:49.548128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 574/1175] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:49 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:49.633637 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f6ad72dc-3eb6-4611-899e-469e400bece5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:49.634243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f6ad72dc-3eb6-4611-899e-469e400bece5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:49.640666 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6ad72dc-3eb6-4611-899e-469e400bece5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:49.641136 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6ad72dc-3eb6-4611-899e-469e400bece5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:49.641904 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6ad72dc-3eb6-4611-899e-469e400bece5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:49.642331 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6ad72dc-3eb6-4611-899e-469e400bece5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:49.771529 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f6ad72dc-3eb6-4611-899e-469e400bece5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:49.779553 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f6ad72dc-3eb6-4611-899e-469e400bece5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149706 Jun 10 13:14:49.780111 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 602/1176] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:49 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:50.559593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8029aad8-336b-49f4-befc-1665cdb8c21a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:50.560574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8029aad8-336b-49f4-befc-1665cdb8c21a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:50.566995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8029aad8-336b-49f4-befc-1665cdb8c21a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:50.567449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8029aad8-336b-49f4-befc-1665cdb8c21a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:50.568198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8029aad8-336b-49f4-befc-1665cdb8c21a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:50.568658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8029aad8-336b-49f4-befc-1665cdb8c21a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:50.597742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8029aad8-336b-49f4-befc-1665cdb8c21a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042546 Jun 10 13:14:50.598220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 575/1177] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:50 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:50.796069 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ccd15b30-e5fb-4881-b180-39bc915f2238 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:50.796662 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ccd15b30-e5fb-4881-b180-39bc915f2238 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:50.803101 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ccd15b30-e5fb-4881-b180-39bc915f2238 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:50.803541 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ccd15b30-e5fb-4881-b180-39bc915f2238 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:50.804293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ccd15b30-e5fb-4881-b180-39bc915f2238 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:50.804717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ccd15b30-e5fb-4881-b180-39bc915f2238 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:50.931206 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ccd15b30-e5fb-4881-b180-39bc915f2238 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:50.939213 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ccd15b30-e5fb-4881-b180-39bc915f2238 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147639 Jun 10 13:14:50.939769 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 603/1178] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:50 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:51.614007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-780e322e-69cb-4fc8-b072-62be96042ffc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:51.614007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-780e322e-69cb-4fc8-b072-62be96042ffc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:51.616569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-780e322e-69cb-4fc8-b072-62be96042ffc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:51.617054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-780e322e-69cb-4fc8-b072-62be96042ffc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:51.617813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-780e322e-69cb-4fc8-b072-62be96042ffc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:51.618248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-780e322e-69cb-4fc8-b072-62be96042ffc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:51.650907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-780e322e-69cb-4fc8-b072-62be96042ffc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046121 Jun 10 13:14:51.651471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 576/1179] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:51 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:51.955664 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6ba54159-ad71-4486-b230-2a1c082d0561 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:51.956268 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6ba54159-ad71-4486-b230-2a1c082d0561 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:51.962765 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6ba54159-ad71-4486-b230-2a1c082d0561 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:51.963212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6ba54159-ad71-4486-b230-2a1c082d0561 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:51.963955 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6ba54159-ad71-4486-b230-2a1c082d0561 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:51.964384 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6ba54159-ad71-4486-b230-2a1c082d0561 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:52.096964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6ba54159-ad71-4486-b230-2a1c082d0561 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:52.104279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6ba54159-ad71-4486-b230-2a1c082d0561 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152437 Jun 10 13:14:52.104851 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 604/1180] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:51 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:52.663350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7ca64ff9-2c75-465e-8eac-e10bec20f698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:52.664203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7ca64ff9-2c75-465e-8eac-e10bec20f698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:52.670632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ca64ff9-2c75-465e-8eac-e10bec20f698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:52.671074 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ca64ff9-2c75-465e-8eac-e10bec20f698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:52.671814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ca64ff9-2c75-465e-8eac-e10bec20f698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:52.672242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ca64ff9-2c75-465e-8eac-e10bec20f698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:52.702075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7ca64ff9-2c75-465e-8eac-e10bec20f698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042776 Jun 10 13:14:52.702618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 577/1181] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:52 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:53.120725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-883411e2-907a-4656-9048-7fee491225b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:53.121367 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-883411e2-907a-4656-9048-7fee491225b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:53.128410 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-883411e2-907a-4656-9048-7fee491225b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:53.128895 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-883411e2-907a-4656-9048-7fee491225b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:53.129654 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-883411e2-907a-4656-9048-7fee491225b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:53.130089 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-883411e2-907a-4656-9048-7fee491225b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:53.264897 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-883411e2-907a-4656-9048-7fee491225b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:53.272371 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-883411e2-907a-4656-9048-7fee491225b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.155506 Jun 10 13:14:53.272952 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 605/1182] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:53 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:53.713637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7a8e3e55-ee90-4df7-97a1-bc5fa871cb88 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:53.714547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7a8e3e55-ee90-4df7-97a1-bc5fa871cb88 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:53.717511 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=60 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:53.722205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a8e3e55-ee90-4df7-97a1-bc5fa871cb88 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:53.722653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a8e3e55-ee90-4df7-97a1-bc5fa871cb88 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:53.723410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a8e3e55-ee90-4df7-97a1-bc5fa871cb88 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:53.723843 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a8e3e55-ee90-4df7-97a1-bc5fa871cb88 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:53.752351 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7a8e3e55-ee90-4df7-97a1-bc5fa871cb88 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042702 Jun 10 13:14:53.752913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 578/1183] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:53 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:54.342798 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9889244e-1ee0-4b54-8edb-8ac0b88ed9a5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:54.343410 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9889244e-1ee0-4b54-8edb-8ac0b88ed9a5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:54.349991 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9889244e-1ee0-4b54-8edb-8ac0b88ed9a5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:54.350617 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9889244e-1ee0-4b54-8edb-8ac0b88ed9a5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:54.351379 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9889244e-1ee0-4b54-8edb-8ac0b88ed9a5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:54.351836 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9889244e-1ee0-4b54-8edb-8ac0b88ed9a5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:54.476366 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9889244e-1ee0-4b54-8edb-8ac0b88ed9a5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:54.484289 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9889244e-1ee0-4b54-8edb-8ac0b88ed9a5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145433 Jun 10 13:14:54.484872 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 606/1184] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:54 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:54.764442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9c980db3-3447-4d66-b1e6-5ec672994e61 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:54.765394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9c980db3-3447-4d66-b1e6-5ec672994e61 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:54.771931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c980db3-3447-4d66-b1e6-5ec672994e61 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:54.772379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c980db3-3447-4d66-b1e6-5ec672994e61 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:54.773167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c980db3-3447-4d66-b1e6-5ec672994e61 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:54.773618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c980db3-3447-4d66-b1e6-5ec672994e61 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:54.802378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9c980db3-3447-4d66-b1e6-5ec672994e61 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041935 Jun 10 13:14:54.802928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 579/1185] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:54 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:55.501615 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-423441a3-8c09-4e2a-8325-94e9200cc843 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:55.502220 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-423441a3-8c09-4e2a-8325-94e9200cc843 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:55.508699 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-423441a3-8c09-4e2a-8325-94e9200cc843 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:55.509168 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-423441a3-8c09-4e2a-8325-94e9200cc843 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:55.509913 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-423441a3-8c09-4e2a-8325-94e9200cc843 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:55.510341 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-423441a3-8c09-4e2a-8325-94e9200cc843 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:55.641175 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-423441a3-8c09-4e2a-8325-94e9200cc843 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:55.648708 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-423441a3-8c09-4e2a-8325-94e9200cc843 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150883 Jun 10 13:14:55.649303 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 607/1186] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:55 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:55.813665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b55d4dce-e1df-4dcd-94a7-fbd49b0e9db8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:55.814577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b55d4dce-e1df-4dcd-94a7-fbd49b0e9db8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:55.821019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b55d4dce-e1df-4dcd-94a7-fbd49b0e9db8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:55.821502 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b55d4dce-e1df-4dcd-94a7-fbd49b0e9db8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:55.822259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b55d4dce-e1df-4dcd-94a7-fbd49b0e9db8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:55.822699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b55d4dce-e1df-4dcd-94a7-fbd49b0e9db8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:55.850858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b55d4dce-e1df-4dcd-94a7-fbd49b0e9db8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041137 Jun 10 13:14:55.851334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 580/1187] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:55 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:56.666261 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2c6e78b3-7176-4188-90e5-b6092491ba38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:56.666838 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2c6e78b3-7176-4188-90e5-b6092491ba38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:56.673861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c6e78b3-7176-4188-90e5-b6092491ba38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:56.674300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c6e78b3-7176-4188-90e5-b6092491ba38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:56.675046 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c6e78b3-7176-4188-90e5-b6092491ba38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:56.675499 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2c6e78b3-7176-4188-90e5-b6092491ba38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:56.802607 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2c6e78b3-7176-4188-90e5-b6092491ba38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:56.810001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2c6e78b3-7176-4188-90e5-b6092491ba38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147630 Jun 10 13:14:56.810560 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 608/1188] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:56 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:56.862654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8f15e1ee-43c2-47d0-9d0a-e1ac519d25f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:56.863576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8f15e1ee-43c2-47d0-9d0a-e1ac519d25f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:56.870284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f15e1ee-43c2-47d0-9d0a-e1ac519d25f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:56.870728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f15e1ee-43c2-47d0-9d0a-e1ac519d25f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:56.871500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f15e1ee-43c2-47d0-9d0a-e1ac519d25f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:56.871919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f15e1ee-43c2-47d0-9d0a-e1ac519d25f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:56.900502 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8f15e1ee-43c2-47d0-9d0a-e1ac519d25f6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042176 Jun 10 13:14:56.901077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 581/1189] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:56 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:57.826855 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cdec1e5b-312e-4c23-a471-c5f3ade6247e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:57.827495 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cdec1e5b-312e-4c23-a471-c5f3ade6247e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:57.833991 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cdec1e5b-312e-4c23-a471-c5f3ade6247e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:57.834429 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cdec1e5b-312e-4c23-a471-c5f3ade6247e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:57.835191 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cdec1e5b-312e-4c23-a471-c5f3ade6247e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:57.835598 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cdec1e5b-312e-4c23-a471-c5f3ade6247e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:57.912805 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6692803b-30ae-4269-8a3b-928bb7312852 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:57.914096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6692803b-30ae-4269-8a3b-928bb7312852 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:57.920565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6692803b-30ae-4269-8a3b-928bb7312852 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:57.921035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6692803b-30ae-4269-8a3b-928bb7312852 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:57.921789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6692803b-30ae-4269-8a3b-928bb7312852 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:57.922226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6692803b-30ae-4269-8a3b-928bb7312852 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:57.950812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6692803b-30ae-4269-8a3b-928bb7312852 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041957 Jun 10 13:14:57.951372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 582/1190] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:57 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:14:57.966785 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cdec1e5b-312e-4c23-a471-c5f3ade6247e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:57.974198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cdec1e5b-312e-4c23-a471-c5f3ade6247e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151814 Jun 10 13:14:57.974754 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 609/1191] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:57 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:14:58.962428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bf67223d-60dc-467c-9ba5-b88921e088ab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:58.963337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bf67223d-60dc-467c-9ba5-b88921e088ab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:58.969750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bf67223d-60dc-467c-9ba5-b88921e088ab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:58.970189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bf67223d-60dc-467c-9ba5-b88921e088ab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:58.970945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bf67223d-60dc-467c-9ba5-b88921e088ab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:58.971378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bf67223d-60dc-467c-9ba5-b88921e088ab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:58.991521 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-643b00e7-7faf-4ce1-9f0a-40dabdbc5cfd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:14:58.992147 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-643b00e7-7faf-4ce1-9f0a-40dabdbc5cfd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:14:58.999106 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-643b00e7-7faf-4ce1-9f0a-40dabdbc5cfd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:58.999544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-643b00e7-7faf-4ce1-9f0a-40dabdbc5cfd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:59.000298 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-643b00e7-7faf-4ce1-9f0a-40dabdbc5cfd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:14:59.000583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bf67223d-60dc-467c-9ba5-b88921e088ab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042122 Jun 10 13:14:59.000714 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-643b00e7-7faf-4ce1-9f0a-40dabdbc5cfd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:14:59.001173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 583/1192] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:14:58 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:14:59.128399 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-643b00e7-7faf-4ce1-9f0a-40dabdbc5cfd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:14:59.135725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-643b00e7-7faf-4ce1-9f0a-40dabdbc5cfd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148042 Jun 10 13:14:59.136285 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 610/1193] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:14:58 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:15:00.011824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0c7bf409-0a11-4b16-a137-7d628cf4e384 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:00.013337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0c7bf409-0a11-4b16-a137-7d628cf4e384 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:00.019828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c7bf409-0a11-4b16-a137-7d628cf4e384 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:00.020267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c7bf409-0a11-4b16-a137-7d628cf4e384 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:00.021062 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c7bf409-0a11-4b16-a137-7d628cf4e384 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:00.021500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c7bf409-0a11-4b16-a137-7d628cf4e384 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:00.049893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0c7bf409-0a11-4b16-a137-7d628cf4e384 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042026 Jun 10 13:15:00.050449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 584/1194] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:00 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:00.154639 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-272e8af5-598f-44fe-ada8-2d5cbf2e2e6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:00.155328 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-272e8af5-598f-44fe-ada8-2d5cbf2e2e6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:00.161801 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-272e8af5-598f-44fe-ada8-2d5cbf2e2e6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:00.162241 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-272e8af5-598f-44fe-ada8-2d5cbf2e2e6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:00.162992 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-272e8af5-598f-44fe-ada8-2d5cbf2e2e6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:00.163419 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-272e8af5-598f-44fe-ada8-2d5cbf2e2e6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:00.291276 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-272e8af5-598f-44fe-ada8-2d5cbf2e2e6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:00.295238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:00.299649 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-272e8af5-598f-44fe-ada8-2d5cbf2e2e6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149531 Jun 10 13:15:00.300279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 611/1195] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:00 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:01.062345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dd782534-2af6-49e8-9638-765b49eca2ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:01.063181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dd782534-2af6-49e8-9638-765b49eca2ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:01.069242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd782534-2af6-49e8-9638-765b49eca2ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:01.069694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd782534-2af6-49e8-9638-765b49eca2ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:01.070454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd782534-2af6-49e8-9638-765b49eca2ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:01.070892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd782534-2af6-49e8-9638-765b49eca2ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:01.099029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dd782534-2af6-49e8-9638-765b49eca2ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041537 Jun 10 13:15:01.099571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 585/1196] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:01 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:01.317233 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8a614fdb-4f1a-4165-8848-418b1fcbabe9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:01.317772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8a614fdb-4f1a-4165-8848-418b1fcbabe9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:01.324659 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8a614fdb-4f1a-4165-8848-418b1fcbabe9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:01.325172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8a614fdb-4f1a-4165-8848-418b1fcbabe9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:01.325925 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8a614fdb-4f1a-4165-8848-418b1fcbabe9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:01.326352 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8a614fdb-4f1a-4165-8848-418b1fcbabe9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:01.454989 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8a614fdb-4f1a-4165-8848-418b1fcbabe9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:01.462396 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8a614fdb-4f1a-4165-8848-418b1fcbabe9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149156 Jun 10 13:15:01.462957 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 612/1197] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:01 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:02.111348 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3fd6df70-3aaa-4f32-ba7c-879cb832926e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:02.112299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3fd6df70-3aaa-4f32-ba7c-879cb832926e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:02.118870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3fd6df70-3aaa-4f32-ba7c-879cb832926e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:02.119317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3fd6df70-3aaa-4f32-ba7c-879cb832926e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:02.120071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3fd6df70-3aaa-4f32-ba7c-879cb832926e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:02.120504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3fd6df70-3aaa-4f32-ba7c-879cb832926e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:02.148704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3fd6df70-3aaa-4f32-ba7c-879cb832926e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041610 Jun 10 13:15:02.149252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 586/1198] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:02 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:02.478837 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-66058159-b84a-4ff5-a2cb-e556a0189412 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:02.479445 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-66058159-b84a-4ff5-a2cb-e556a0189412 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:02.482339 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=52 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:02.487326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66058159-b84a-4ff5-a2cb-e556a0189412 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:02.487772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66058159-b84a-4ff5-a2cb-e556a0189412 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:02.488526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66058159-b84a-4ff5-a2cb-e556a0189412 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:02.488990 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66058159-b84a-4ff5-a2cb-e556a0189412 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:02.616796 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-66058159-b84a-4ff5-a2cb-e556a0189412 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:02.624729 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-66058159-b84a-4ff5-a2cb-e556a0189412 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149708 Jun 10 13:15:02.625313 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 613/1199] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:02 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:03.160122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-25a79c50-5984-4786-90d4-4445966b69bf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:03.161051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-25a79c50-5984-4786-90d4-4445966b69bf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:03.167478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25a79c50-5984-4786-90d4-4445966b69bf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:03.167915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25a79c50-5984-4786-90d4-4445966b69bf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:03.169368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25a79c50-5984-4786-90d4-4445966b69bf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:03.169801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25a79c50-5984-4786-90d4-4445966b69bf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:03.182965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:03.198049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-25a79c50-5984-4786-90d4-4445966b69bf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041872 Jun 10 13:15:03.198638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 587/1200] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:03 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:03.641840 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-34a7d351-aa2a-49b1-8584-c92f95c74991 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:03.642398 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-34a7d351-aa2a-49b1-8584-c92f95c74991 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:03.648926 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-34a7d351-aa2a-49b1-8584-c92f95c74991 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:03.649371 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-34a7d351-aa2a-49b1-8584-c92f95c74991 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:03.650120 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-34a7d351-aa2a-49b1-8584-c92f95c74991 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:03.650551 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-34a7d351-aa2a-49b1-8584-c92f95c74991 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:03.826291 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-34a7d351-aa2a-49b1-8584-c92f95c74991 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:03.833667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-34a7d351-aa2a-49b1-8584-c92f95c74991 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.196423 Jun 10 13:15:03.834228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 614/1201] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:03 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:04.210943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-98964a50-5b77-4ca1-be07-e8daa15de436 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:04.211853 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-98964a50-5b77-4ca1-be07-e8daa15de436 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:04.218764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98964a50-5b77-4ca1-be07-e8daa15de436 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:04.219215 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98964a50-5b77-4ca1-be07-e8daa15de436 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:04.219985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98964a50-5b77-4ca1-be07-e8daa15de436 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:04.220423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98964a50-5b77-4ca1-be07-e8daa15de436 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:04.250463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-98964a50-5b77-4ca1-be07-e8daa15de436 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043915 Jun 10 13:15:04.251019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 588/1202] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:04 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:04.850690 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-85fa7407-93d3-4cc0-954d-407dc2dd6650 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:04.851328 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-85fa7407-93d3-4cc0-954d-407dc2dd6650 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:04.858445 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-85fa7407-93d3-4cc0-954d-407dc2dd6650 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:04.858881 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-85fa7407-93d3-4cc0-954d-407dc2dd6650 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:04.859626 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-85fa7407-93d3-4cc0-954d-407dc2dd6650 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:04.860050 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-85fa7407-93d3-4cc0-954d-407dc2dd6650 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:04.985488 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-85fa7407-93d3-4cc0-954d-407dc2dd6650 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:04.993043 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-85fa7407-93d3-4cc0-954d-407dc2dd6650 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146182 Jun 10 13:15:04.993602 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 615/1203] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:04 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:05.262444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-21e9dd3a-a1ed-41d0-81e7-cc405be046f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:05.263354 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-21e9dd3a-a1ed-41d0-81e7-cc405be046f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:05.269900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-21e9dd3a-a1ed-41d0-81e7-cc405be046f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:05.270349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-21e9dd3a-a1ed-41d0-81e7-cc405be046f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:05.271101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-21e9dd3a-a1ed-41d0-81e7-cc405be046f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:05.271535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-21e9dd3a-a1ed-41d0-81e7-cc405be046f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:05.299653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-21e9dd3a-a1ed-41d0-81e7-cc405be046f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041925 Jun 10 13:15:05.300188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 589/1204] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:05 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:05.303465 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:15:05.303623 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Fri Jun 10 13:15:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:15:06.010482 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-285e4f4e-b7d2-4303-9e37-dd0783ab04ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:06.011102 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-285e4f4e-b7d2-4303-9e37-dd0783ab04ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:06.018158 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-285e4f4e-b7d2-4303-9e37-dd0783ab04ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:06.018602 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-285e4f4e-b7d2-4303-9e37-dd0783ab04ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:06.019362 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-285e4f4e-b7d2-4303-9e37-dd0783ab04ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:06.019792 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-285e4f4e-b7d2-4303-9e37-dd0783ab04ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:06.145650 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-285e4f4e-b7d2-4303-9e37-dd0783ab04ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:06.153040 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-285e4f4e-b7d2-4303-9e37-dd0783ab04ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146394 Jun 10 13:15:06.153597 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 616/1205] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:06 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:06.312003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c6afac33-7998-42d3-b1aa-2f656d57a22e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:06.312937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c6afac33-7998-42d3-b1aa-2f656d57a22e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:06.319433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6afac33-7998-42d3-b1aa-2f656d57a22e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:06.319879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6afac33-7998-42d3-b1aa-2f656d57a22e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:06.320630 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6afac33-7998-42d3-b1aa-2f656d57a22e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:06.321087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6afac33-7998-42d3-b1aa-2f656d57a22e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:06.349726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c6afac33-7998-42d3-b1aa-2f656d57a22e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042281 Jun 10 13:15:06.350210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 590/1206] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:06 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:07.170348 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-32f48392-bb41-4ae6-a351-db997d642c70 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:07.170964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-32f48392-bb41-4ae6-a351-db997d642c70 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:07.177153 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-32f48392-bb41-4ae6-a351-db997d642c70 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:07.177604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-32f48392-bb41-4ae6-a351-db997d642c70 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:07.183511 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-32f48392-bb41-4ae6-a351-db997d642c70 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:07.183511 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-32f48392-bb41-4ae6-a351-db997d642c70 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:07.314706 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-32f48392-bb41-4ae6-a351-db997d642c70 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:07.322348 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-32f48392-bb41-4ae6-a351-db997d642c70 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.156430 Jun 10 13:15:07.322896 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 617/1207] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:07 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:07.361340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-130585a1-2341-41d7-a19c-b0691c0ec075 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:07.362258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-130585a1-2341-41d7-a19c-b0691c0ec075 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:07.368672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-130585a1-2341-41d7-a19c-b0691c0ec075 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:07.369141 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-130585a1-2341-41d7-a19c-b0691c0ec075 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:07.369888 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-130585a1-2341-41d7-a19c-b0691c0ec075 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:07.370316 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-130585a1-2341-41d7-a19c-b0691c0ec075 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:07.398828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-130585a1-2341-41d7-a19c-b0691c0ec075 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041728 Jun 10 13:15:07.399366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 591/1208] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:07 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:08.339739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5a9ce2a9-2fc1-4101-aada-571144e8fe59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:08.340994 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5a9ce2a9-2fc1-4101-aada-571144e8fe59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:08.347571 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a9ce2a9-2fc1-4101-aada-571144e8fe59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:08.348015 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a9ce2a9-2fc1-4101-aada-571144e8fe59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:08.348820 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a9ce2a9-2fc1-4101-aada-571144e8fe59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:08.349255 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a9ce2a9-2fc1-4101-aada-571144e8fe59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:08.411065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-710ee3f4-2e63-4fc3-8548-f5b607170bcb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:08.411977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-710ee3f4-2e63-4fc3-8548-f5b607170bcb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:08.418383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-710ee3f4-2e63-4fc3-8548-f5b607170bcb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:08.418827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-710ee3f4-2e63-4fc3-8548-f5b607170bcb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:08.419589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-710ee3f4-2e63-4fc3-8548-f5b607170bcb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:08.420019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-710ee3f4-2e63-4fc3-8548-f5b607170bcb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:08.449837 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-710ee3f4-2e63-4fc3-8548-f5b607170bcb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042175 Jun 10 13:15:08.450514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 592/1209] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:08 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:15:08.480692 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5a9ce2a9-2fc1-4101-aada-571144e8fe59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:08.488507 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5a9ce2a9-2fc1-4101-aada-571144e8fe59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152701 Jun 10 13:15:08.489097 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 618/1210] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:08 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:09.461085 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a3e01b6c-f414-4abd-9884-0da6de33e605 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:09.462005 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a3e01b6c-f414-4abd-9884-0da6de33e605 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:09.469080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3e01b6c-f414-4abd-9884-0da6de33e605 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:09.469531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3e01b6c-f414-4abd-9884-0da6de33e605 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:09.470282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3e01b6c-f414-4abd-9884-0da6de33e605 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:09.470711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3e01b6c-f414-4abd-9884-0da6de33e605 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:09.498566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a3e01b6c-f414-4abd-9884-0da6de33e605 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041474 Jun 10 13:15:09.499103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 593/1211] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:09 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:09.505849 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-47094cde-dd07-4bf7-aa45-cc03eb945a24 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:09.506461 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-47094cde-dd07-4bf7-aa45-cc03eb945a24 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:09.513063 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-47094cde-dd07-4bf7-aa45-cc03eb945a24 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:09.513511 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-47094cde-dd07-4bf7-aa45-cc03eb945a24 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:09.514262 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-47094cde-dd07-4bf7-aa45-cc03eb945a24 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:09.514691 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-47094cde-dd07-4bf7-aa45-cc03eb945a24 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:09.643153 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-47094cde-dd07-4bf7-aa45-cc03eb945a24 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:09.650563 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-47094cde-dd07-4bf7-aa45-cc03eb945a24 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148542 Jun 10 13:15:09.651154 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 619/1212] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:09 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:10.510328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3bdb128b-2971-469c-9251-9bf6d6aca2b8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:10.511261 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3bdb128b-2971-469c-9251-9bf6d6aca2b8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:10.517724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3bdb128b-2971-469c-9251-9bf6d6aca2b8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:10.518170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3bdb128b-2971-469c-9251-9bf6d6aca2b8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:10.518922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3bdb128b-2971-469c-9251-9bf6d6aca2b8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:10.519351 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3bdb128b-2971-469c-9251-9bf6d6aca2b8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:10.547389 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3bdb128b-2971-469c-9251-9bf6d6aca2b8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041066 Jun 10 13:15:10.547922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 594/1213] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:10 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:10.668858 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3c964cde-9cb5-494a-afd4-93fdc89078f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:10.669392 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3c964cde-9cb5-494a-afd4-93fdc89078f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:10.675823 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c964cde-9cb5-494a-afd4-93fdc89078f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:10.676266 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c964cde-9cb5-494a-afd4-93fdc89078f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:10.677045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c964cde-9cb5-494a-afd4-93fdc89078f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:10.677482 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c964cde-9cb5-494a-afd4-93fdc89078f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:10.806873 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3c964cde-9cb5-494a-afd4-93fdc89078f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:10.814418 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3c964cde-9cb5-494a-afd4-93fdc89078f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150098 Jun 10 13:15:10.814975 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 620/1214] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:10 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:11.559414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f4cd9319-dcda-4733-b46e-fe58dbc25e8b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:11.560313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f4cd9319-dcda-4733-b46e-fe58dbc25e8b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:11.566706 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f4cd9319-dcda-4733-b46e-fe58dbc25e8b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:11.567149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f4cd9319-dcda-4733-b46e-fe58dbc25e8b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:11.567900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f4cd9319-dcda-4733-b46e-fe58dbc25e8b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:11.568328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f4cd9319-dcda-4733-b46e-fe58dbc25e8b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:11.596684 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f4cd9319-dcda-4733-b46e-fe58dbc25e8b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041705 Jun 10 13:15:11.597194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 595/1215] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:11 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:11.831583 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-48d08b58-1bc2-4ec8-8ed6-b935a20f1464 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:11.832191 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-48d08b58-1bc2-4ec8-8ed6-b935a20f1464 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:11.839665 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-48d08b58-1bc2-4ec8-8ed6-b935a20f1464 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:11.840672 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-48d08b58-1bc2-4ec8-8ed6-b935a20f1464 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:11.841458 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-48d08b58-1bc2-4ec8-8ed6-b935a20f1464 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:11.841914 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-48d08b58-1bc2-4ec8-8ed6-b935a20f1464 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:11.970853 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-48d08b58-1bc2-4ec8-8ed6-b935a20f1464 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:11.978842 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-48d08b58-1bc2-4ec8-8ed6-b935a20f1464 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151103 Jun 10 13:15:11.979406 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 621/1216] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:11 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:12.609653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-738701d5-4b3f-4b7e-a4bb-8d8464a3af84 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:12.610535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-738701d5-4b3f-4b7e-a4bb-8d8464a3af84 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:12.618222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-738701d5-4b3f-4b7e-a4bb-8d8464a3af84 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:12.618671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-738701d5-4b3f-4b7e-a4bb-8d8464a3af84 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:12.619434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-738701d5-4b3f-4b7e-a4bb-8d8464a3af84 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:12.619867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-738701d5-4b3f-4b7e-a4bb-8d8464a3af84 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:12.649146 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-738701d5-4b3f-4b7e-a4bb-8d8464a3af84 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044354 Jun 10 13:15:12.649694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 596/1217] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:12 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:13.000210 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6908bb36-96ae-4397-b49b-f17f0140f76a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:13.000944 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6908bb36-96ae-4397-b49b-f17f0140f76a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:13.007806 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6908bb36-96ae-4397-b49b-f17f0140f76a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:13.008248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6908bb36-96ae-4397-b49b-f17f0140f76a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:13.009029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6908bb36-96ae-4397-b49b-f17f0140f76a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:13.009465 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6908bb36-96ae-4397-b49b-f17f0140f76a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:13.141036 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6908bb36-96ae-4397-b49b-f17f0140f76a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:13.148572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6908bb36-96ae-4397-b49b-f17f0140f76a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152977 Jun 10 13:15:13.149158 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 622/1218] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:12 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:13.726716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c76245e0-5e81-4fd6-972a-d6b12bceacb1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:13.731015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c76245e0-5e81-4fd6-972a-d6b12bceacb1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:13.733632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c76245e0-5e81-4fd6-972a-d6b12bceacb1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:13.734085 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c76245e0-5e81-4fd6-972a-d6b12bceacb1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:13.734843 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c76245e0-5e81-4fd6-972a-d6b12bceacb1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:13.735280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c76245e0-5e81-4fd6-972a-d6b12bceacb1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:13.764358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c76245e0-5e81-4fd6-972a-d6b12bceacb1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.107632 Jun 10 13:15:13.764928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 597/1219] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:13 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:14.166080 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-529442e6-7a6f-4e6e-a116-583d04e15ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:14.166683 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-529442e6-7a6f-4e6e-a116-583d04e15ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:14.173268 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-529442e6-7a6f-4e6e-a116-583d04e15ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:14.173710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-529442e6-7a6f-4e6e-a116-583d04e15ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:14.174452 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-529442e6-7a6f-4e6e-a116-583d04e15ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:14.174880 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-529442e6-7a6f-4e6e-a116-583d04e15ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:14.183248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:15:14.183396 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:15:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:15:14.302368 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-529442e6-7a6f-4e6e-a116-583d04e15ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:14.309817 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-529442e6-7a6f-4e6e-a116-583d04e15ead tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147502 Jun 10 13:15:14.310298 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 623/1220] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:14 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:14.776277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-07526a06-7204-4bc2-baf4-826925a7942b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:14.777825 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-07526a06-7204-4bc2-baf4-826925a7942b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:14.784195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07526a06-7204-4bc2-baf4-826925a7942b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:14.784633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07526a06-7204-4bc2-baf4-826925a7942b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:14.785417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07526a06-7204-4bc2-baf4-826925a7942b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:14.785843 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07526a06-7204-4bc2-baf4-826925a7942b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:14.813859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-07526a06-7204-4bc2-baf4-826925a7942b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041798 Jun 10 13:15:14.814389 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 598/1221] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:14 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:15.327231 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8d2dc175-57c1-4482-8c4a-aa9dbd3aa45e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:15.327838 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8d2dc175-57c1-4482-8c4a-aa9dbd3aa45e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:15.334357 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8d2dc175-57c1-4482-8c4a-aa9dbd3aa45e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:15.334809 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8d2dc175-57c1-4482-8c4a-aa9dbd3aa45e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:15.335572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8d2dc175-57c1-4482-8c4a-aa9dbd3aa45e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:15.336403 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8d2dc175-57c1-4482-8c4a-aa9dbd3aa45e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:15.466700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8d2dc175-57c1-4482-8c4a-aa9dbd3aa45e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:15.474514 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8d2dc175-57c1-4482-8c4a-aa9dbd3aa45e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151743 Jun 10 13:15:15.475075 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 624/1222] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:15 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:15.826237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-639b99e8-3c62-4822-b5d0-80b744202549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:15.827156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-639b99e8-3c62-4822-b5d0-80b744202549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:15.833724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-639b99e8-3c62-4822-b5d0-80b744202549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:15.834169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-639b99e8-3c62-4822-b5d0-80b744202549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:15.834924 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-639b99e8-3c62-4822-b5d0-80b744202549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:15.835384 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-639b99e8-3c62-4822-b5d0-80b744202549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:15.864006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-639b99e8-3c62-4822-b5d0-80b744202549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042589 Jun 10 13:15:15.864545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 599/1223] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:15 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:16.491585 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8154e92e-0578-4b30-8555-3bf40b74a0e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:16.492123 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8154e92e-0578-4b30-8555-3bf40b74a0e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:16.499231 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8154e92e-0578-4b30-8555-3bf40b74a0e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:16.499737 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8154e92e-0578-4b30-8555-3bf40b74a0e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:16.500433 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8154e92e-0578-4b30-8555-3bf40b74a0e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:16.500895 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8154e92e-0578-4b30-8555-3bf40b74a0e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:16.704561 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8154e92e-0578-4b30-8555-3bf40b74a0e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:16.712182 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8154e92e-0578-4b30-8555-3bf40b74a0e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.224432 Jun 10 13:15:16.712753 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 625/1224] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:16 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:16.876380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e9a86714-8188-474b-b908-3f6db85c007a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:16.877335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e9a86714-8188-474b-b908-3f6db85c007a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:16.884685 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9a86714-8188-474b-b908-3f6db85c007a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:16.885161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9a86714-8188-474b-b908-3f6db85c007a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:16.885921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9a86714-8188-474b-b908-3f6db85c007a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:16.886424 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9a86714-8188-474b-b908-3f6db85c007a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:16.916123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e9a86714-8188-474b-b908-3f6db85c007a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043772 Jun 10 13:15:16.916677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 600/1225] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:16 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:15:17.729226 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-46486627-e6ba-413b-8c1b-eaf38b90ccac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:17.729912 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-46486627-e6ba-413b-8c1b-eaf38b90ccac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:17.736950 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46486627-e6ba-413b-8c1b-eaf38b90ccac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:17.737463 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46486627-e6ba-413b-8c1b-eaf38b90ccac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:17.738356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46486627-e6ba-413b-8c1b-eaf38b90ccac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:17.738875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46486627-e6ba-413b-8c1b-eaf38b90ccac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:17.869716 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-46486627-e6ba-413b-8c1b-eaf38b90ccac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:17.877170 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-46486627-e6ba-413b-8c1b-eaf38b90ccac tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151654 Jun 10 13:15:17.877726 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 626/1226] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:17 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:17.928156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4382a16b-8713-4e30-9414-39054d22ec97 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:17.929094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4382a16b-8713-4e30-9414-39054d22ec97 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:17.935651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4382a16b-8713-4e30-9414-39054d22ec97 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:17.936078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4382a16b-8713-4e30-9414-39054d22ec97 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:17.936857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4382a16b-8713-4e30-9414-39054d22ec97 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:17.937295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4382a16b-8713-4e30-9414-39054d22ec97 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:17.965985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4382a16b-8713-4e30-9414-39054d22ec97 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041732 Jun 10 13:15:17.966442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 601/1227] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:17 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:18.894527 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-684fe0ad-cc7e-47cc-a895-b3466311ddee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:18.895125 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-684fe0ad-cc7e-47cc-a895-b3466311ddee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:18.901627 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-684fe0ad-cc7e-47cc-a895-b3466311ddee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:18.902607 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-684fe0ad-cc7e-47cc-a895-b3466311ddee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:18.903351 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-684fe0ad-cc7e-47cc-a895-b3466311ddee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:18.903773 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-684fe0ad-cc7e-47cc-a895-b3466311ddee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:18.977691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0dd6422d-a9cc-47b7-8e44-dac74a171525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:18.978604 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0dd6422d-a9cc-47b7-8e44-dac74a171525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:18.985974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0dd6422d-a9cc-47b7-8e44-dac74a171525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:18.986420 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0dd6422d-a9cc-47b7-8e44-dac74a171525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:18.987176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0dd6422d-a9cc-47b7-8e44-dac74a171525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:18.987611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0dd6422d-a9cc-47b7-8e44-dac74a171525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:19.016742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0dd6422d-a9cc-47b7-8e44-dac74a171525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043147 Jun 10 13:15:19.017327 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 602/1228] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:18 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:15:19.029656 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-684fe0ad-cc7e-47cc-a895-b3466311ddee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:19.037338 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-684fe0ad-cc7e-47cc-a895-b3466311ddee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146673 Jun 10 13:15:19.037894 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 627/1229] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:18 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:20.027680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5df0c82c-0094-41c1-9de2-2153b93c8ed4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:20.029373 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5df0c82c-0094-41c1-9de2-2153b93c8ed4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:20.035922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5df0c82c-0094-41c1-9de2-2153b93c8ed4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:20.036370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5df0c82c-0094-41c1-9de2-2153b93c8ed4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:20.037178 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5df0c82c-0094-41c1-9de2-2153b93c8ed4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:20.037616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5df0c82c-0094-41c1-9de2-2153b93c8ed4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:20.054183 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7357c9fd-c3d9-4e18-9df3-869b530f1948 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:20.054792 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7357c9fd-c3d9-4e18-9df3-869b530f1948 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:20.061489 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7357c9fd-c3d9-4e18-9df3-869b530f1948 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:20.061934 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7357c9fd-c3d9-4e18-9df3-869b530f1948 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:20.062702 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7357c9fd-c3d9-4e18-9df3-869b530f1948 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:20.063133 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7357c9fd-c3d9-4e18-9df3-869b530f1948 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:20.066753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5df0c82c-0094-41c1-9de2-2153b93c8ed4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043048 Jun 10 13:15:20.067317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 603/1230] 10.222.0.25 () {60 vars in 1342 bytes} [Fri Jun 10 13:15:20 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jun 10 13:15:20.082778 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:20.191127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7357c9fd-c3d9-4e18-9df3-869b530f1948 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:20.199259 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7357c9fd-c3d9-4e18-9df3-869b530f1948 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148353 Jun 10 13:15:20.199833 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 628/1231] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:20 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:15:21.078435 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-39a8d945-86f1-4579-9821-2de93c41658d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:15:21.079385 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-39a8d945-86f1-4579-9821-2de93c41658d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:21.085787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39a8d945-86f1-4579-9821-2de93c41658d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:21.086238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39a8d945-86f1-4579-9821-2de93c41658d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:21.086992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39a8d945-86f1-4579-9821-2de93c41658d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:21.087422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39a8d945-86f1-4579-9821-2de93c41658d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:21.091465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:21.195513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-39a8d945-86f1-4579-9821-2de93c41658d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "POST /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/action" status: 200 len: 37767 microversion: 2.20 time: 0.120834 Jun 10 13:15:21.196423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 604/1232] 10.222.0.25 () {62 vars in 1318 bytes} [Fri Jun 10 13:15:21 2022] POST /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3/action => generated 37767 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 361 bytes (1 switches on core 0) Jun 10 13:15:21.215624 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cbf1bc27-00d4-451c-b21c-a1770d93f79d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:21.216237 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cbf1bc27-00d4-451c-b21c-a1770d93f79d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:21.222787 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbf1bc27-00d4-451c-b21c-a1770d93f79d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:21.223233 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbf1bc27-00d4-451c-b21c-a1770d93f79d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:21.223986 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbf1bc27-00d4-451c-b21c-a1770d93f79d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:21.224416 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbf1bc27-00d4-451c-b21c-a1770d93f79d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:21.360167 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cbf1bc27-00d4-451c-b21c-a1770d93f79d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:21.367895 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cbf1bc27-00d4-451c-b21c-a1770d93f79d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.156595 Jun 10 13:15:21.368453 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 629/1233] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:21 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:22.386174 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7a595f8b-b540-48a7-9e06-ff1cce2825c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:22.386904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7a595f8b-b540-48a7-9e06-ff1cce2825c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:22.394216 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a595f8b-b540-48a7-9e06-ff1cce2825c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:22.394663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a595f8b-b540-48a7-9e06-ff1cce2825c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:22.395416 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a595f8b-b540-48a7-9e06-ff1cce2825c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:22.395849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a595f8b-b540-48a7-9e06-ff1cce2825c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:22.573133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7a595f8b-b540-48a7-9e06-ff1cce2825c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:22.580623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7a595f8b-b540-48a7-9e06-ff1cce2825c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.198656 Jun 10 13:15:22.581212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 605/1234] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:22 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:23.598323 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-19b57c6a-3ee9-4e77-bb9c-17893f2db416 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:23.598848 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-19b57c6a-3ee9-4e77-bb9c-17893f2db416 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:23.606086 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-19b57c6a-3ee9-4e77-bb9c-17893f2db416 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:23.607390 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-19b57c6a-3ee9-4e77-bb9c-17893f2db416 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:23.607390 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-19b57c6a-3ee9-4e77-bb9c-17893f2db416 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:23.607822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-19b57c6a-3ee9-4e77-bb9c-17893f2db416 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:23.745122 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-19b57c6a-3ee9-4e77-bb9c-17893f2db416 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:23.749631 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-19b57c6a-3ee9-4e77-bb9c-17893f2db416 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.155337 Jun 10 13:15:23.750197 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 630/1235] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:23 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:24.766929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-20e6e3a5-ef37-4685-b90d-024f7d3c9534 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:24.767537 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-20e6e3a5-ef37-4685-b90d-024f7d3c9534 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:24.769961 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=56 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:24.774983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20e6e3a5-ef37-4685-b90d-024f7d3c9534 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:24.775433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20e6e3a5-ef37-4685-b90d-024f7d3c9534 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:24.776310 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20e6e3a5-ef37-4685-b90d-024f7d3c9534 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:24.776667 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20e6e3a5-ef37-4685-b90d-024f7d3c9534 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:24.908136 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-20e6e3a5-ef37-4685-b90d-024f7d3c9534 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:24.915538 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-20e6e3a5-ef37-4685-b90d-024f7d3c9534 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152531 Jun 10 13:15:24.916093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 606/1236] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:24 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:25.933226 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a9a3a1a0-e82b-486c-a545-ccb3c3fd11ef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:25.933859 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a9a3a1a0-e82b-486c-a545-ccb3c3fd11ef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:25.941099 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a9a3a1a0-e82b-486c-a545-ccb3c3fd11ef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:25.941538 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a9a3a1a0-e82b-486c-a545-ccb3c3fd11ef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:25.942285 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a9a3a1a0-e82b-486c-a545-ccb3c3fd11ef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:25.942710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a9a3a1a0-e82b-486c-a545-ccb3c3fd11ef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:26.073351 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a9a3a1a0-e82b-486c-a545-ccb3c3fd11ef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:26.080977 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a9a3a1a0-e82b-486c-a545-ccb3c3fd11ef tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151517 Jun 10 13:15:26.081542 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 631/1237] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:25 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:27.098062 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-930ab82c-b619-430a-a07e-3fa9de103780 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:27.098671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-930ab82c-b619-430a-a07e-3fa9de103780 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:27.105283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-930ab82c-b619-430a-a07e-3fa9de103780 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:27.105728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-930ab82c-b619-430a-a07e-3fa9de103780 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:27.106486 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-930ab82c-b619-430a-a07e-3fa9de103780 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:27.106919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-930ab82c-b619-430a-a07e-3fa9de103780 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:27.233943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-930ab82c-b619-430a-a07e-3fa9de103780 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:27.241727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-930ab82c-b619-430a-a07e-3fa9de103780 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147498 Jun 10 13:15:27.242285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 607/1238] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:27 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:28.258169 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-13207b89-b4c8-4a94-bc75-6b38bb90ff69 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:28.258777 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-13207b89-b4c8-4a94-bc75-6b38bb90ff69 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:28.265778 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-13207b89-b4c8-4a94-bc75-6b38bb90ff69 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:28.266220 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-13207b89-b4c8-4a94-bc75-6b38bb90ff69 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:28.266972 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-13207b89-b4c8-4a94-bc75-6b38bb90ff69 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:28.267401 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-13207b89-b4c8-4a94-bc75-6b38bb90ff69 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:28.393112 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-13207b89-b4c8-4a94-bc75-6b38bb90ff69 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:28.400559 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-13207b89-b4c8-4a94-bc75-6b38bb90ff69 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146223 Jun 10 13:15:28.401162 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 632/1239] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:28 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:29.418331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dcb68123-988f-4be2-8c48-696867039157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:29.419575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dcb68123-988f-4be2-8c48-696867039157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:29.426112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcb68123-988f-4be2-8c48-696867039157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:29.426556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcb68123-988f-4be2-8c48-696867039157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:29.427310 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcb68123-988f-4be2-8c48-696867039157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:29.427741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcb68123-988f-4be2-8c48-696867039157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:29.554584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dcb68123-988f-4be2-8c48-696867039157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:29.562478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dcb68123-988f-4be2-8c48-696867039157 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147993 Jun 10 13:15:29.563037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 608/1240] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:29 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:30.579277 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3171eca2-1783-49ab-9475-dc7665608caf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:30.579879 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3171eca2-1783-49ab-9475-dc7665608caf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:30.586431 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3171eca2-1783-49ab-9475-dc7665608caf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:30.586820 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3171eca2-1783-49ab-9475-dc7665608caf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:30.587563 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3171eca2-1783-49ab-9475-dc7665608caf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:30.587985 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3171eca2-1783-49ab-9475-dc7665608caf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:30.717055 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3171eca2-1783-49ab-9475-dc7665608caf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:30.724369 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3171eca2-1783-49ab-9475-dc7665608caf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148951 Jun 10 13:15:30.724955 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 633/1241] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:30 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:31.741635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-10e90da4-02f5-4b61-ae26-9842983c5a20 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:31.742300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-10e90da4-02f5-4b61-ae26-9842983c5a20 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:31.748689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10e90da4-02f5-4b61-ae26-9842983c5a20 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:31.749164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10e90da4-02f5-4b61-ae26-9842983c5a20 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:31.749913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10e90da4-02f5-4b61-ae26-9842983c5a20 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:31.750346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10e90da4-02f5-4b61-ae26-9842983c5a20 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:31.878182 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10e90da4-02f5-4b61-ae26-9842983c5a20 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:31.885651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-10e90da4-02f5-4b61-ae26-9842983c5a20 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147985 Jun 10 13:15:31.890650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 609/1242] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:31 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:32.908217 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d1e6c02a-9759-4680-ad14-e7ee5df58edd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:32.908837 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d1e6c02a-9759-4680-ad14-e7ee5df58edd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:32.911119 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=42 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:32.915773 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d1e6c02a-9759-4680-ad14-e7ee5df58edd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:32.916215 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d1e6c02a-9759-4680-ad14-e7ee5df58edd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:32.917002 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d1e6c02a-9759-4680-ad14-e7ee5df58edd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:32.917434 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d1e6c02a-9759-4680-ad14-e7ee5df58edd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:33.044452 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d1e6c02a-9759-4680-ad14-e7ee5df58edd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:33.051939 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d1e6c02a-9759-4680-ad14-e7ee5df58edd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147577 Jun 10 13:15:33.055773 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 634/1243] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:32 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:34.069759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-878a655d-44ce-4697-8218-05337cc4c66b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:34.070403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-878a655d-44ce-4697-8218-05337cc4c66b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:34.077516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-878a655d-44ce-4697-8218-05337cc4c66b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:34.077967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-878a655d-44ce-4697-8218-05337cc4c66b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:34.078729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-878a655d-44ce-4697-8218-05337cc4c66b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:34.079163 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-878a655d-44ce-4697-8218-05337cc4c66b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:34.203024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-878a655d-44ce-4697-8218-05337cc4c66b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:34.210457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-878a655d-44ce-4697-8218-05337cc4c66b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.144531 Jun 10 13:15:34.211014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 610/1244] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:34 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:35.227406 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-61ebef13-d597-41bc-9aa7-560fb2c4cf06 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:35.228014 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-61ebef13-d597-41bc-9aa7-560fb2c4cf06 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:35.234593 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-61ebef13-d597-41bc-9aa7-560fb2c4cf06 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:35.235038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-61ebef13-d597-41bc-9aa7-560fb2c4cf06 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:35.236092 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-61ebef13-d597-41bc-9aa7-560fb2c4cf06 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:35.236523 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-61ebef13-d597-41bc-9aa7-560fb2c4cf06 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:35.361730 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-61ebef13-d597-41bc-9aa7-560fb2c4cf06 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:35.369171 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-61ebef13-d597-41bc-9aa7-560fb2c4cf06 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145549 Jun 10 13:15:35.369738 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 635/1245] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:35 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:36.390581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a6efb9fe-5121-49f0-a87d-0a014f5f1ea2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:36.391188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a6efb9fe-5121-49f0-a87d-0a014f5f1ea2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:36.397730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a6efb9fe-5121-49f0-a87d-0a014f5f1ea2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:36.398178 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a6efb9fe-5121-49f0-a87d-0a014f5f1ea2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:36.398936 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a6efb9fe-5121-49f0-a87d-0a014f5f1ea2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:36.399370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a6efb9fe-5121-49f0-a87d-0a014f5f1ea2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:36.527236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a6efb9fe-5121-49f0-a87d-0a014f5f1ea2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:36.534653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a6efb9fe-5121-49f0-a87d-0a014f5f1ea2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148569 Jun 10 13:15:36.535215 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 611/1246] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:36 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:37.552206 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-29ce1815-2c89-41eb-946d-0868755ea30c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:37.552717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-29ce1815-2c89-41eb-946d-0868755ea30c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:37.559756 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29ce1815-2c89-41eb-946d-0868755ea30c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:37.560206 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29ce1815-2c89-41eb-946d-0868755ea30c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:37.561000 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29ce1815-2c89-41eb-946d-0868755ea30c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:37.561465 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-29ce1815-2c89-41eb-946d-0868755ea30c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:37.689961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-29ce1815-2c89-41eb-946d-0868755ea30c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:37.697487 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-29ce1815-2c89-41eb-946d-0868755ea30c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149174 Jun 10 13:15:37.698057 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 636/1247] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:37 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:38.714811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f7627d6f-4a8e-47f2-907e-c78d685beb60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:38.716049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f7627d6f-4a8e-47f2-907e-c78d685beb60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:38.722368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f7627d6f-4a8e-47f2-907e-c78d685beb60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:38.722814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f7627d6f-4a8e-47f2-907e-c78d685beb60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:38.723569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f7627d6f-4a8e-47f2-907e-c78d685beb60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:38.724058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f7627d6f-4a8e-47f2-907e-c78d685beb60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:38.852932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f7627d6f-4a8e-47f2-907e-c78d685beb60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:38.860638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f7627d6f-4a8e-47f2-907e-c78d685beb60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149399 Jun 10 13:15:38.861227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 612/1248] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:38 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:39.878306 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-14e14eef-0e0a-42e5-95ca-534c707f7cf9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:39.878908 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-14e14eef-0e0a-42e5-95ca-534c707f7cf9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:39.885018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14e14eef-0e0a-42e5-95ca-534c707f7cf9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:39.885451 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14e14eef-0e0a-42e5-95ca-534c707f7cf9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:39.886210 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14e14eef-0e0a-42e5-95ca-534c707f7cf9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:39.886619 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14e14eef-0e0a-42e5-95ca-534c707f7cf9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:40.016482 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-14e14eef-0e0a-42e5-95ca-534c707f7cf9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:40.024183 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-14e14eef-0e0a-42e5-95ca-534c707f7cf9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149901 Jun 10 13:15:40.024790 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 637/1249] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:39 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:41.041629 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-acbb5e39-25f7-456c-b30a-5833c83d524c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:41.042161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-acbb5e39-25f7-456c-b30a-5833c83d524c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:41.048611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acbb5e39-25f7-456c-b30a-5833c83d524c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:41.049091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acbb5e39-25f7-456c-b30a-5833c83d524c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:41.049851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acbb5e39-25f7-456c-b30a-5833c83d524c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:41.050285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acbb5e39-25f7-456c-b30a-5833c83d524c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:41.178129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-acbb5e39-25f7-456c-b30a-5833c83d524c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:41.185632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-acbb5e39-25f7-456c-b30a-5833c83d524c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147986 Jun 10 13:15:41.186195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 613/1250] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:41 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:42.202352 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-47ad17e8-04d9-4224-aa89-804804e26328 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:42.202955 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-47ad17e8-04d9-4224-aa89-804804e26328 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:42.209457 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-47ad17e8-04d9-4224-aa89-804804e26328 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:42.209924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-47ad17e8-04d9-4224-aa89-804804e26328 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:42.210677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-47ad17e8-04d9-4224-aa89-804804e26328 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:42.211102 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-47ad17e8-04d9-4224-aa89-804804e26328 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:42.582858 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-47ad17e8-04d9-4224-aa89-804804e26328 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:42.591078 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-47ad17e8-04d9-4224-aa89-804804e26328 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.391903 Jun 10 13:15:42.591579 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 638/1251] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:42 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:43.611645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d812f5e6-9f7c-4e43-9a52-e63eaa80c0d3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:43.611645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d812f5e6-9f7c-4e43-9a52-e63eaa80c0d3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:43.615499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d812f5e6-9f7c-4e43-9a52-e63eaa80c0d3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:43.615944 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d812f5e6-9f7c-4e43-9a52-e63eaa80c0d3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:43.616700 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d812f5e6-9f7c-4e43-9a52-e63eaa80c0d3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:43.617157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d812f5e6-9f7c-4e43-9a52-e63eaa80c0d3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:43.745328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d812f5e6-9f7c-4e43-9a52-e63eaa80c0d3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:43.752817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d812f5e6-9f7c-4e43-9a52-e63eaa80c0d3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149257 Jun 10 13:15:43.754988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 614/1252] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:43 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:44.769798 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2d5f1a0e-ddf5-4385-a9d3-1f112fb24dbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:44.770405 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2d5f1a0e-ddf5-4385-a9d3-1f112fb24dbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:44.776902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d5f1a0e-ddf5-4385-a9d3-1f112fb24dbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:44.777349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d5f1a0e-ddf5-4385-a9d3-1f112fb24dbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:44.778100 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d5f1a0e-ddf5-4385-a9d3-1f112fb24dbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:44.778531 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d5f1a0e-ddf5-4385-a9d3-1f112fb24dbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:44.903532 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2d5f1a0e-ddf5-4385-a9d3-1f112fb24dbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:44.911232 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2d5f1a0e-ddf5-4385-a9d3-1f112fb24dbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145262 Jun 10 13:15:44.911792 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 639/1253] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:44 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:15:45.929127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c47fe081-a522-4c15-ba3f-bef1f2e31ce7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:45.929733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c47fe081-a522-4c15-ba3f-bef1f2e31ce7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:45.936153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c47fe081-a522-4c15-ba3f-bef1f2e31ce7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:45.936586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c47fe081-a522-4c15-ba3f-bef1f2e31ce7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:45.937371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c47fe081-a522-4c15-ba3f-bef1f2e31ce7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:45.937798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c47fe081-a522-4c15-ba3f-bef1f2e31ce7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:46.062244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c47fe081-a522-4c15-ba3f-bef1f2e31ce7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:46.069689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c47fe081-a522-4c15-ba3f-bef1f2e31ce7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.144507 Jun 10 13:15:46.070250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 615/1254] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:45 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:47.086783 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-04f5f884-6c4d-4052-b53e-a4781ac7dc6a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:47.087394 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-04f5f884-6c4d-4052-b53e-a4781ac7dc6a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:47.094620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04f5f884-6c4d-4052-b53e-a4781ac7dc6a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:47.095061 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04f5f884-6c4d-4052-b53e-a4781ac7dc6a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:47.095843 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04f5f884-6c4d-4052-b53e-a4781ac7dc6a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:47.096274 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04f5f884-6c4d-4052-b53e-a4781ac7dc6a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:47.225620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-04f5f884-6c4d-4052-b53e-a4781ac7dc6a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:47.234209 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-04f5f884-6c4d-4052-b53e-a4781ac7dc6a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151204 Jun 10 13:15:47.234791 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 640/1255] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:47 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:48.251634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4cf5d438-dd29-4a7b-8bd6-6b8092d8348c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:48.252250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4cf5d438-dd29-4a7b-8bd6-6b8092d8348c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:48.259140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4cf5d438-dd29-4a7b-8bd6-6b8092d8348c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:48.259832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4cf5d438-dd29-4a7b-8bd6-6b8092d8348c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:48.260665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4cf5d438-dd29-4a7b-8bd6-6b8092d8348c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:48.261191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4cf5d438-dd29-4a7b-8bd6-6b8092d8348c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:48.412339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4cf5d438-dd29-4a7b-8bd6-6b8092d8348c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:48.420449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4cf5d438-dd29-4a7b-8bd6-6b8092d8348c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.172701 Jun 10 13:15:48.421039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 616/1256] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:48 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:49.446671 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7ce1d144-ef8b-4cc8-ab9f-9f3f408b7cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:49.447287 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7ce1d144-ef8b-4cc8-ab9f-9f3f408b7cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:49.453883 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7ce1d144-ef8b-4cc8-ab9f-9f3f408b7cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:49.454330 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7ce1d144-ef8b-4cc8-ab9f-9f3f408b7cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:49.455090 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7ce1d144-ef8b-4cc8-ab9f-9f3f408b7cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:49.455520 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7ce1d144-ef8b-4cc8-ab9f-9f3f408b7cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:49.582618 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7ce1d144-ef8b-4cc8-ab9f-9f3f408b7cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:49.590660 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7ce1d144-ef8b-4cc8-ab9f-9f3f408b7cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147346 Jun 10 13:15:49.591240 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 641/1257] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:49 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:50.607415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-37a168ee-fdf1-44da-8a61-4c5872fa1cab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:50.607949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-37a168ee-fdf1-44da-8a61-4c5872fa1cab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:50.614780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-37a168ee-fdf1-44da-8a61-4c5872fa1cab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:50.615222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-37a168ee-fdf1-44da-8a61-4c5872fa1cab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:50.615973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-37a168ee-fdf1-44da-8a61-4c5872fa1cab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:50.616402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-37a168ee-fdf1-44da-8a61-4c5872fa1cab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:50.741108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-37a168ee-fdf1-44da-8a61-4c5872fa1cab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:50.749004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-37a168ee-fdf1-44da-8a61-4c5872fa1cab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145457 Jun 10 13:15:50.749563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 617/1258] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:50 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:51.766492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3cd7ce13-0890-4fb1-939b-c86b55f20683 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:51.767116 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3cd7ce13-0890-4fb1-939b-c86b55f20683 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:51.773586 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3cd7ce13-0890-4fb1-939b-c86b55f20683 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:51.777688 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3cd7ce13-0890-4fb1-939b-c86b55f20683 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:51.777688 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3cd7ce13-0890-4fb1-939b-c86b55f20683 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:51.777688 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3cd7ce13-0890-4fb1-939b-c86b55f20683 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:51.777985 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=81 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:51.903449 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3cd7ce13-0890-4fb1-939b-c86b55f20683 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:51.910968 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3cd7ce13-0890-4fb1-939b-c86b55f20683 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148899 Jun 10 13:15:51.911528 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 642/1259] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:51 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:52.927340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cf35063f-ce8f-4bf9-990a-e87aa1915a02 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:52.927981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cf35063f-ce8f-4bf9-990a-e87aa1915a02 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:52.934560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf35063f-ce8f-4bf9-990a-e87aa1915a02 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:52.935009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf35063f-ce8f-4bf9-990a-e87aa1915a02 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:52.935760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf35063f-ce8f-4bf9-990a-e87aa1915a02 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:52.936189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf35063f-ce8f-4bf9-990a-e87aa1915a02 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:52.938728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=82 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:53.067164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cf35063f-ce8f-4bf9-990a-e87aa1915a02 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:53.075133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cf35063f-ce8f-4bf9-990a-e87aa1915a02 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151150 Jun 10 13:15:53.076836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 618/1260] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:52 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:54.093016 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-69990384-f2e5-45e1-a402-3d440714dd9c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:54.093626 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-69990384-f2e5-45e1-a402-3d440714dd9c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:54.100173 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69990384-f2e5-45e1-a402-3d440714dd9c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:54.100615 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69990384-f2e5-45e1-a402-3d440714dd9c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:54.101415 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69990384-f2e5-45e1-a402-3d440714dd9c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:54.101848 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69990384-f2e5-45e1-a402-3d440714dd9c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:54.231099 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-69990384-f2e5-45e1-a402-3d440714dd9c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:54.238827 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-69990384-f2e5-45e1-a402-3d440714dd9c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150217 Jun 10 13:15:54.239403 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 643/1261] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:54 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:55.255792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cfc00e2a-9b50-424e-ac64-fbbdb6805484 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:55.256394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cfc00e2a-9b50-424e-ac64-fbbdb6805484 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:55.258739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=26 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:55.263607 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfc00e2a-9b50-424e-ac64-fbbdb6805484 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:55.264057 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfc00e2a-9b50-424e-ac64-fbbdb6805484 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:55.264832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfc00e2a-9b50-424e-ac64-fbbdb6805484 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:55.265260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfc00e2a-9b50-424e-ac64-fbbdb6805484 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:55.395960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cfc00e2a-9b50-424e-ac64-fbbdb6805484 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:55.403315 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cfc00e2a-9b50-424e-ac64-fbbdb6805484 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151380 Jun 10 13:15:55.403876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 619/1262] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:55 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:56.420226 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9f99b454-6dfa-40b0-939b-577fb75388f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:56.420880 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9f99b454-6dfa-40b0-939b-577fb75388f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:56.427325 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9f99b454-6dfa-40b0-939b-577fb75388f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:56.427774 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9f99b454-6dfa-40b0-939b-577fb75388f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:56.428535 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9f99b454-6dfa-40b0-939b-577fb75388f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:56.428992 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9f99b454-6dfa-40b0-939b-577fb75388f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:56.552163 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9f99b454-6dfa-40b0-939b-577fb75388f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:56.560249 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9f99b454-6dfa-40b0-939b-577fb75388f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.143848 Jun 10 13:15:56.560822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 644/1263] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:56 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:57.577368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3c9ee0da-b091-4903-bcae-0bd67fe6e3c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:57.578007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3c9ee0da-b091-4903-bcae-0bd67fe6e3c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:57.587673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c9ee0da-b091-4903-bcae-0bd67fe6e3c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:57.588124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c9ee0da-b091-4903-bcae-0bd67fe6e3c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:57.588974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c9ee0da-b091-4903-bcae-0bd67fe6e3c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:57.589414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c9ee0da-b091-4903-bcae-0bd67fe6e3c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:57.724107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3c9ee0da-b091-4903-bcae-0bd67fe6e3c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:57.732381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3c9ee0da-b091-4903-bcae-0bd67fe6e3c5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.158908 Jun 10 13:15:57.732975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 620/1264] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:57 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:58.749527 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4272f211-0a54-4702-8d5a-131fb5d041d5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:58.750117 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4272f211-0a54-4702-8d5a-131fb5d041d5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:58.757022 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4272f211-0a54-4702-8d5a-131fb5d041d5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:58.758131 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4272f211-0a54-4702-8d5a-131fb5d041d5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:58.759023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4272f211-0a54-4702-8d5a-131fb5d041d5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:58.760019 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4272f211-0a54-4702-8d5a-131fb5d041d5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:58.892859 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4272f211-0a54-4702-8d5a-131fb5d041d5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:15:58.900219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4272f211-0a54-4702-8d5a-131fb5d041d5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.154666 Jun 10 13:15:58.900795 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 645/1265] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:58 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:15:59.917801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-892368f1-4a01-4266-95fa-a74a85fc03bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:15:59.921585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-892368f1-4a01-4266-95fa-a74a85fc03bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:15:59.925003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-892368f1-4a01-4266-95fa-a74a85fc03bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:59.925448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-892368f1-4a01-4266-95fa-a74a85fc03bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:15:59.926198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-892368f1-4a01-4266-95fa-a74a85fc03bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:15:59.926628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-892368f1-4a01-4266-95fa-a74a85fc03bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:00.052052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-892368f1-4a01-4266-95fa-a74a85fc03bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:00.060152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-892368f1-4a01-4266-95fa-a74a85fc03bf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146248 Jun 10 13:16:00.060710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 621/1266] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:15:59 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:01.077374 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b37f2351-4107-46fe-9f50-d051b7fd422d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:01.077937 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b37f2351-4107-46fe-9f50-d051b7fd422d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:01.084373 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b37f2351-4107-46fe-9f50-d051b7fd422d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:01.084839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b37f2351-4107-46fe-9f50-d051b7fd422d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:01.085585 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b37f2351-4107-46fe-9f50-d051b7fd422d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:01.086014 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b37f2351-4107-46fe-9f50-d051b7fd422d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:01.211838 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b37f2351-4107-46fe-9f50-d051b7fd422d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:01.219957 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b37f2351-4107-46fe-9f50-d051b7fd422d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146538 Jun 10 13:16:01.220512 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 646/1267] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:01 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:02.239149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fce67992-da04-4332-bdd6-ef50fc781110 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:02.239765 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fce67992-da04-4332-bdd6-ef50fc781110 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:02.246451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fce67992-da04-4332-bdd6-ef50fc781110 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:02.246906 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fce67992-da04-4332-bdd6-ef50fc781110 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:02.247663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fce67992-da04-4332-bdd6-ef50fc781110 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:02.248098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fce67992-da04-4332-bdd6-ef50fc781110 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:02.374205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fce67992-da04-4332-bdd6-ef50fc781110 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:02.381627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fce67992-da04-4332-bdd6-ef50fc781110 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148246 Jun 10 13:16:02.382192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 622/1268] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:02 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:03.398464 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ee02588f-475c-45c1-8363-eb88f6534d19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:03.399066 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ee02588f-475c-45c1-8363-eb88f6534d19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:03.401381 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=26 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:16:03.406142 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ee02588f-475c-45c1-8363-eb88f6534d19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:03.406587 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ee02588f-475c-45c1-8363-eb88f6534d19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:03.407373 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ee02588f-475c-45c1-8363-eb88f6534d19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:03.407805 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ee02588f-475c-45c1-8363-eb88f6534d19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:03.534514 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ee02588f-475c-45c1-8363-eb88f6534d19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:03.541935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ee02588f-475c-45c1-8363-eb88f6534d19 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147365 Jun 10 13:16:03.542501 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 647/1269] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:03 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:04.559945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-54c7b228-18ba-4235-91cd-84a20a10973e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:04.560549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-54c7b228-18ba-4235-91cd-84a20a10973e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:04.566971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-54c7b228-18ba-4235-91cd-84a20a10973e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:04.567423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-54c7b228-18ba-4235-91cd-84a20a10973e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:04.568175 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-54c7b228-18ba-4235-91cd-84a20a10973e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:04.568608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-54c7b228-18ba-4235-91cd-84a20a10973e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:04.694968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-54c7b228-18ba-4235-91cd-84a20a10973e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:04.702394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-54c7b228-18ba-4235-91cd-84a20a10973e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146811 Jun 10 13:16:04.702948 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 623/1270] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:04 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:05.719438 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4cc58d39-2dee-4009-9454-a69e7d177a59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:05.720049 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4cc58d39-2dee-4009-9454-a69e7d177a59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:05.726552 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4cc58d39-2dee-4009-9454-a69e7d177a59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:05.727001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4cc58d39-2dee-4009-9454-a69e7d177a59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:05.727755 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4cc58d39-2dee-4009-9454-a69e7d177a59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:05.728190 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4cc58d39-2dee-4009-9454-a69e7d177a59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:05.857272 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4cc58d39-2dee-4009-9454-a69e7d177a59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:05.864969 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4cc58d39-2dee-4009-9454-a69e7d177a59 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149418 Jun 10 13:16:05.865540 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 648/1271] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:05 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:06.882508 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2ccc7fab-8780-4403-91eb-96dc3801e282 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:06.883118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2ccc7fab-8780-4403-91eb-96dc3801e282 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:06.889593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ccc7fab-8780-4403-91eb-96dc3801e282 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:06.890040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ccc7fab-8780-4403-91eb-96dc3801e282 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:06.890797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ccc7fab-8780-4403-91eb-96dc3801e282 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:06.891226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ccc7fab-8780-4403-91eb-96dc3801e282 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:07.017877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2ccc7fab-8780-4403-91eb-96dc3801e282 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:07.024724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2ccc7fab-8780-4403-91eb-96dc3801e282 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146131 Jun 10 13:16:07.025314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 624/1272] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:06 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:08.042209 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-968f2bde-95b7-4f65-a11e-90a5747b86e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:08.043358 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-968f2bde-95b7-4f65-a11e-90a5747b86e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:08.049758 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-968f2bde-95b7-4f65-a11e-90a5747b86e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:08.050213 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-968f2bde-95b7-4f65-a11e-90a5747b86e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:08.050954 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-968f2bde-95b7-4f65-a11e-90a5747b86e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:08.051386 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-968f2bde-95b7-4f65-a11e-90a5747b86e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:08.176088 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-968f2bde-95b7-4f65-a11e-90a5747b86e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:08.184181 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-968f2bde-95b7-4f65-a11e-90a5747b86e2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145270 Jun 10 13:16:08.184758 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 649/1273] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:08 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:09.201838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1a09ac51-7715-4ec8-b350-26ed4896ffe7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:09.202466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1a09ac51-7715-4ec8-b350-26ed4896ffe7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:09.208947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a09ac51-7715-4ec8-b350-26ed4896ffe7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:09.209389 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a09ac51-7715-4ec8-b350-26ed4896ffe7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:09.210133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a09ac51-7715-4ec8-b350-26ed4896ffe7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:09.210559 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a09ac51-7715-4ec8-b350-26ed4896ffe7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:09.341108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1a09ac51-7715-4ec8-b350-26ed4896ffe7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:09.348589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1a09ac51-7715-4ec8-b350-26ed4896ffe7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151191 Jun 10 13:16:09.349176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 625/1274] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:09 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:10.366361 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3dad64d1-2467-4908-84d5-6584c07ab040 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:10.366970 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3dad64d1-2467-4908-84d5-6584c07ab040 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:10.373701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3dad64d1-2467-4908-84d5-6584c07ab040 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:10.374154 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3dad64d1-2467-4908-84d5-6584c07ab040 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:10.374908 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3dad64d1-2467-4908-84d5-6584c07ab040 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:10.375341 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3dad64d1-2467-4908-84d5-6584c07ab040 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:10.503142 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3dad64d1-2467-4908-84d5-6584c07ab040 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:10.510496 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3dad64d1-2467-4908-84d5-6584c07ab040 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148208 Jun 10 13:16:10.511085 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 650/1275] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:10 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:11.527384 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b072e08b-85a0-4b61-9676-2df7286e3ab5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:11.527983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b072e08b-85a0-4b61-9676-2df7286e3ab5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:11.534440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b072e08b-85a0-4b61-9676-2df7286e3ab5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:11.534884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b072e08b-85a0-4b61-9676-2df7286e3ab5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:11.535660 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b072e08b-85a0-4b61-9676-2df7286e3ab5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:11.536088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b072e08b-85a0-4b61-9676-2df7286e3ab5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:11.660179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b072e08b-85a0-4b61-9676-2df7286e3ab5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:11.668234 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b072e08b-85a0-4b61-9676-2df7286e3ab5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.144239 Jun 10 13:16:11.668814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 626/1276] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:11 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:12.685953 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-59123a1d-83ee-4fe3-87e6-6dfc548d7b30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:12.686478 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-59123a1d-83ee-4fe3-87e6-6dfc548d7b30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:12.693017 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59123a1d-83ee-4fe3-87e6-6dfc548d7b30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:12.693466 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59123a1d-83ee-4fe3-87e6-6dfc548d7b30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:12.694219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59123a1d-83ee-4fe3-87e6-6dfc548d7b30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:12.694648 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-59123a1d-83ee-4fe3-87e6-6dfc548d7b30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:12.819142 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-59123a1d-83ee-4fe3-87e6-6dfc548d7b30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:12.826639 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-59123a1d-83ee-4fe3-87e6-6dfc548d7b30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145332 Jun 10 13:16:12.827197 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 651/1277] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:12 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:13.843462 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-35e9b17a-cc70-49e4-90fd-3c29f428142e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:13.844062 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-35e9b17a-cc70-49e4-90fd-3c29f428142e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:13.850602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35e9b17a-cc70-49e4-90fd-3c29f428142e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:13.851042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35e9b17a-cc70-49e4-90fd-3c29f428142e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:13.851791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35e9b17a-cc70-49e4-90fd-3c29f428142e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:13.852217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35e9b17a-cc70-49e4-90fd-3c29f428142e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:13.976644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-35e9b17a-cc70-49e4-90fd-3c29f428142e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:13.984547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-35e9b17a-cc70-49e4-90fd-3c29f428142e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.144990 Jun 10 13:16:13.985384 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 627/1278] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:13 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:15.001050 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e9e2a5f2-1d03-45eb-a0d7-84ff70a75b8d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:15.001648 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e9e2a5f2-1d03-45eb-a0d7-84ff70a75b8d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:15.008039 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9e2a5f2-1d03-45eb-a0d7-84ff70a75b8d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:15.008480 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9e2a5f2-1d03-45eb-a0d7-84ff70a75b8d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:15.009251 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9e2a5f2-1d03-45eb-a0d7-84ff70a75b8d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:15.009679 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9e2a5f2-1d03-45eb-a0d7-84ff70a75b8d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:15.114053 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e13a9757-eaec-40f1-b7c4-16d8eb4a8147 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:15.114553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e13a9757-eaec-40f1-b7c4-16d8eb4a8147 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:15.121703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e13a9757-eaec-40f1-b7c4-16d8eb4a8147 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:15.122375 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e13a9757-eaec-40f1-b7c4-16d8eb4a8147 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:15.123369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e13a9757-eaec-40f1-b7c4-16d8eb4a8147 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:15.123806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e13a9757-eaec-40f1-b7c4-16d8eb4a8147 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:15.159773 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e9e2a5f2-1d03-45eb-a0d7-84ff70a75b8d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:15.161015 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e9e2a5f2-1d03-45eb-a0d7-84ff70a75b8d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.163779 Jun 10 13:16:15.161577 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 652/1279] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:14 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:16.178260 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cd46cee0-6fd7-46b8-8a5e-0063abd17e51 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:16.178897 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cd46cee0-6fd7-46b8-8a5e-0063abd17e51 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:16.185799 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cd46cee0-6fd7-46b8-8a5e-0063abd17e51 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:16.186249 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cd46cee0-6fd7-46b8-8a5e-0063abd17e51 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:16.187027 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cd46cee0-6fd7-46b8-8a5e-0063abd17e51 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:16.187464 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cd46cee0-6fd7-46b8-8a5e-0063abd17e51 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:16.315024 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cd46cee0-6fd7-46b8-8a5e-0063abd17e51 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:16.322490 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cd46cee0-6fd7-46b8-8a5e-0063abd17e51 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148170 Jun 10 13:16:16.323087 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 653/1280] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:16 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:16.328336 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-480c733d-961b-42e3-8745-dd0e2b9ccbea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:16.328854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-480c733d-961b-42e3-8745-dd0e2b9ccbea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:16.334953 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-480c733d-961b-42e3-8745-dd0e2b9ccbea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:16.335340 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-480c733d-961b-42e3-8745-dd0e2b9ccbea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:16.336552 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-480c733d-961b-42e3-8745-dd0e2b9ccbea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:16.336552 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-480c733d-961b-42e3-8745-dd0e2b9ccbea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:16.361601 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-480c733d-961b-42e3-8745-dd0e2b9ccbea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Going to try to terminate instance {{(pid=118093) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:16:16.623596 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-480c733d-961b-42e3-8745-dd0e2b9ccbea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 204 len: 0 microversion: 2.1 time: 0.298678 Jun 10 13:16:16.624250 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 654/1281] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:16:16 2022] DELETE /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jun 10 13:16:16.632426 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e6b4112d-632f-46b6-bbba-74562e6fb6bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:16.635571 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e6b4112d-632f-46b6-bbba-74562e6fb6bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:16.639970 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6b4112d-632f-46b6-bbba-74562e6fb6bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:16.640414 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6b4112d-632f-46b6-bbba-74562e6fb6bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:16.641242 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6b4112d-632f-46b6-bbba-74562e6fb6bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:16.641670 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6b4112d-632f-46b6-bbba-74562e6fb6bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:16.770629 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e6b4112d-632f-46b6-bbba-74562e6fb6bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:16.778609 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e6b4112d-632f-46b6-bbba-74562e6fb6bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.150077 Jun 10 13:16:16.779177 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 655/1282] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:16 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:17.340336 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5aca5bf9-7f03-4c0c-b39f-8c6e15b69c67 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:17.341068 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5aca5bf9-7f03-4c0c-b39f-8c6e15b69c67 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:17.348251 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5aca5bf9-7f03-4c0c-b39f-8c6e15b69c67 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:17.348692 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5aca5bf9-7f03-4c0c-b39f-8c6e15b69c67 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:17.349494 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5aca5bf9-7f03-4c0c-b39f-8c6e15b69c67 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:17.349922 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5aca5bf9-7f03-4c0c-b39f-8c6e15b69c67 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:17.536599 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5aca5bf9-7f03-4c0c-b39f-8c6e15b69c67 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:17.545690 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5aca5bf9-7f03-4c0c-b39f-8c6e15b69c67 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.209884 Jun 10 13:16:17.546266 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 656/1283] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:17 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:17.795417 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-65b370a3-3ab0-4f1f-91b8-f79f643b5548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:17.796016 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-65b370a3-3ab0-4f1f-91b8-f79f643b5548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:17.801920 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-65b370a3-3ab0-4f1f-91b8-f79f643b5548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:17.802363 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-65b370a3-3ab0-4f1f-91b8-f79f643b5548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:17.803109 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-65b370a3-3ab0-4f1f-91b8-f79f643b5548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:17.803537 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-65b370a3-3ab0-4f1f-91b8-f79f643b5548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:17.807504 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:16:17.929355 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-65b370a3-3ab0-4f1f-91b8-f79f643b5548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:17.937427 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-65b370a3-3ab0-4f1f-91b8-f79f643b5548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145751 Jun 10 13:16:17.937991 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 657/1284] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:17 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:18.567710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a4b3ab09-634b-4e9a-8240-cdd33900a037 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:18.568378 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a4b3ab09-634b-4e9a-8240-cdd33900a037 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:18.575782 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a4b3ab09-634b-4e9a-8240-cdd33900a037 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:18.576221 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a4b3ab09-634b-4e9a-8240-cdd33900a037 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:18.577008 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a4b3ab09-634b-4e9a-8240-cdd33900a037 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:18.577441 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a4b3ab09-634b-4e9a-8240-cdd33900a037 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:18.713176 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a4b3ab09-634b-4e9a-8240-cdd33900a037 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:18.721237 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a4b3ab09-634b-4e9a-8240-cdd33900a037 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.157868 Jun 10 13:16:18.721806 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 658/1285] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:18 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:18.955204 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a4c461eb-e01f-457c-8b79-97e77529d764 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:18.955813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a4c461eb-e01f-457c-8b79-97e77529d764 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:18.962081 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a4c461eb-e01f-457c-8b79-97e77529d764 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:18.962531 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a4c461eb-e01f-457c-8b79-97e77529d764 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:18.963283 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a4c461eb-e01f-457c-8b79-97e77529d764 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:18.963713 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a4c461eb-e01f-457c-8b79-97e77529d764 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:19.092785 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a4c461eb-e01f-457c-8b79-97e77529d764 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:19.102087 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a4c461eb-e01f-457c-8b79-97e77529d764 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.149402 Jun 10 13:16:19.103100 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 659/1286] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:18 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:19.737941 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-82755d3d-c3b9-49b1-8bf0-ab5096a1aad2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:19.738575 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-82755d3d-c3b9-49b1-8bf0-ab5096a1aad2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:19.745050 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-82755d3d-c3b9-49b1-8bf0-ab5096a1aad2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:19.745498 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-82755d3d-c3b9-49b1-8bf0-ab5096a1aad2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:19.746250 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-82755d3d-c3b9-49b1-8bf0-ab5096a1aad2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:19.746681 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-82755d3d-c3b9-49b1-8bf0-ab5096a1aad2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:19.877223 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-82755d3d-c3b9-49b1-8bf0-ab5096a1aad2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:19.884658 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-82755d3d-c3b9-49b1-8bf0-ab5096a1aad2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150813 Jun 10 13:16:19.885252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 660/1287] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:19 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:20.119533 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-42a4569e-e8ca-408d-84d1-33cf3144f95d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:20.120160 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-42a4569e-e8ca-408d-84d1-33cf3144f95d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:20.126193 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-42a4569e-e8ca-408d-84d1-33cf3144f95d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:20.126631 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-42a4569e-e8ca-408d-84d1-33cf3144f95d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:20.127376 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-42a4569e-e8ca-408d-84d1-33cf3144f95d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:20.127801 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-42a4569e-e8ca-408d-84d1-33cf3144f95d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:20.251231 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-42a4569e-e8ca-408d-84d1-33cf3144f95d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:20.259298 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-42a4569e-e8ca-408d-84d1-33cf3144f95d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143570 Jun 10 13:16:20.259856 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 661/1288] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:20 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:20.901680 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3ee77f70-701f-4690-87c7-5482cbdaa7c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:20.902343 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3ee77f70-701f-4690-87c7-5482cbdaa7c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:20.908972 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ee77f70-701f-4690-87c7-5482cbdaa7c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:20.909414 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ee77f70-701f-4690-87c7-5482cbdaa7c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:20.910177 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ee77f70-701f-4690-87c7-5482cbdaa7c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:20.910608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ee77f70-701f-4690-87c7-5482cbdaa7c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:21.039579 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3ee77f70-701f-4690-87c7-5482cbdaa7c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:21.047580 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3ee77f70-701f-4690-87c7-5482cbdaa7c9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149760 Jun 10 13:16:21.048141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 662/1289] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:20 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:21.277045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cbe858d3-f35f-4d9c-8de8-f36812f86800 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:21.277653 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cbe858d3-f35f-4d9c-8de8-f36812f86800 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:21.283721 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbe858d3-f35f-4d9c-8de8-f36812f86800 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:21.284168 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbe858d3-f35f-4d9c-8de8-f36812f86800 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:21.284954 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbe858d3-f35f-4d9c-8de8-f36812f86800 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:21.285389 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbe858d3-f35f-4d9c-8de8-f36812f86800 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:21.410573 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cbe858d3-f35f-4d9c-8de8-f36812f86800 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:21.418909 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cbe858d3-f35f-4d9c-8de8-f36812f86800 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146185 Jun 10 13:16:21.419474 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 663/1290] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:21 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:22.065168 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f97aa4b2-98c2-4a24-aa1c-41941c1e9d35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:22.065787 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f97aa4b2-98c2-4a24-aa1c-41941c1e9d35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:22.072186 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f97aa4b2-98c2-4a24-aa1c-41941c1e9d35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:22.072633 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f97aa4b2-98c2-4a24-aa1c-41941c1e9d35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:22.074092 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f97aa4b2-98c2-4a24-aa1c-41941c1e9d35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:22.074528 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f97aa4b2-98c2-4a24-aa1c-41941c1e9d35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:22.201089 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f97aa4b2-98c2-4a24-aa1c-41941c1e9d35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:22.208574 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f97aa4b2-98c2-4a24-aa1c-41941c1e9d35 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147246 Jun 10 13:16:22.209203 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 664/1291] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:22 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:22.436015 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b83b1a14-8c01-4b6c-ac5e-defafa5a07f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:22.436651 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b83b1a14-8c01-4b6c-ac5e-defafa5a07f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:22.442650 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b83b1a14-8c01-4b6c-ac5e-defafa5a07f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:22.443097 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b83b1a14-8c01-4b6c-ac5e-defafa5a07f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:22.443850 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b83b1a14-8c01-4b6c-ac5e-defafa5a07f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:22.444280 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b83b1a14-8c01-4b6c-ac5e-defafa5a07f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:22.570096 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b83b1a14-8c01-4b6c-ac5e-defafa5a07f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:22.577977 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b83b1a14-8c01-4b6c-ac5e-defafa5a07f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146507 Jun 10 13:16:22.578545 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 665/1292] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:22 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:23.226242 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-db5a1b46-1994-47d9-a605-755c1d6e20f3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:23.226897 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-db5a1b46-1994-47d9-a605-755c1d6e20f3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:23.233936 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-db5a1b46-1994-47d9-a605-755c1d6e20f3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:23.234385 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-db5a1b46-1994-47d9-a605-755c1d6e20f3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:23.235394 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-db5a1b46-1994-47d9-a605-755c1d6e20f3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:23.235840 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-db5a1b46-1994-47d9-a605-755c1d6e20f3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:23.365366 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-db5a1b46-1994-47d9-a605-755c1d6e20f3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:23.372911 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-db5a1b46-1994-47d9-a605-755c1d6e20f3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150472 Jun 10 13:16:23.373504 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 666/1293] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:23 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:23.595472 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b1e3ea0f-e05a-432e-b5f1-58c5c0d1fd83 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:23.596003 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b1e3ea0f-e05a-432e-b5f1-58c5c0d1fd83 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:23.601970 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b1e3ea0f-e05a-432e-b5f1-58c5c0d1fd83 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:23.602408 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b1e3ea0f-e05a-432e-b5f1-58c5c0d1fd83 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:23.603161 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b1e3ea0f-e05a-432e-b5f1-58c5c0d1fd83 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:23.603590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b1e3ea0f-e05a-432e-b5f1-58c5c0d1fd83 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:23.732047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b1e3ea0f-e05a-432e-b5f1-58c5c0d1fd83 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:23.740499 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b1e3ea0f-e05a-432e-b5f1-58c5c0d1fd83 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148952 Jun 10 13:16:23.741091 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 667/1294] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:23 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:24.344637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:16:24.345124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:16:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e (ip 10.222.0.25) !!! Jun 10 13:16:24.390324 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d4dc7bb4-80f3-4a1a-9f93-db9a55402b29 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:24.390927 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d4dc7bb4-80f3-4a1a-9f93-db9a55402b29 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:24.397328 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d4dc7bb4-80f3-4a1a-9f93-db9a55402b29 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:24.397803 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d4dc7bb4-80f3-4a1a-9f93-db9a55402b29 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:24.398544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d4dc7bb4-80f3-4a1a-9f93-db9a55402b29 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:24.398970 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d4dc7bb4-80f3-4a1a-9f93-db9a55402b29 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:24.525790 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d4dc7bb4-80f3-4a1a-9f93-db9a55402b29 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:24.533162 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d4dc7bb4-80f3-4a1a-9f93-db9a55402b29 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147257 Jun 10 13:16:24.533718 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 668/1295] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:24 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:24.757409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f55536a1-9487-4a53-87d8-9bfb2c53c389 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:24.758021 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f55536a1-9487-4a53-87d8-9bfb2c53c389 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:24.763930 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f55536a1-9487-4a53-87d8-9bfb2c53c389 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:24.764372 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f55536a1-9487-4a53-87d8-9bfb2c53c389 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:24.765176 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f55536a1-9487-4a53-87d8-9bfb2c53c389 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:24.765606 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f55536a1-9487-4a53-87d8-9bfb2c53c389 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:24.892062 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f55536a1-9487-4a53-87d8-9bfb2c53c389 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:24.899890 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f55536a1-9487-4a53-87d8-9bfb2c53c389 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146266 Jun 10 13:16:24.900448 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 669/1296] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:24 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:25.119518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=19 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:16:25.193263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=63 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:16:25.549901 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8e7ff775-a824-4132-9621-3e7b97ad7490 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:25.550492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8e7ff775-a824-4132-9621-3e7b97ad7490 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:25.557011 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8e7ff775-a824-4132-9621-3e7b97ad7490 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:25.557456 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8e7ff775-a824-4132-9621-3e7b97ad7490 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:25.558710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8e7ff775-a824-4132-9621-3e7b97ad7490 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:25.559145 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8e7ff775-a824-4132-9621-3e7b97ad7490 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:25.695020 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8e7ff775-a824-4132-9621-3e7b97ad7490 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:25.703017 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8e7ff775-a824-4132-9621-3e7b97ad7490 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.156994 Jun 10 13:16:25.703584 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 670/1297] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:25 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:25.917941 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2e98f10b-6942-4344-8611-c6b9295feb3f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:25.918472 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2e98f10b-6942-4344-8611-c6b9295feb3f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:25.924344 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2e98f10b-6942-4344-8611-c6b9295feb3f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:25.924808 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2e98f10b-6942-4344-8611-c6b9295feb3f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:25.925564 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2e98f10b-6942-4344-8611-c6b9295feb3f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:25.925994 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2e98f10b-6942-4344-8611-c6b9295feb3f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:26.060022 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2e98f10b-6942-4344-8611-c6b9295feb3f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:26.067938 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2e98f10b-6942-4344-8611-c6b9295feb3f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.154380 Jun 10 13:16:26.068499 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 671/1298] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:25 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:26.720493 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-26efb686-ed24-4065-be40-dfc9dc2fccaa tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:26.721155 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-26efb686-ed24-4065-be40-dfc9dc2fccaa tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:26.727716 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26efb686-ed24-4065-be40-dfc9dc2fccaa tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:26.728159 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26efb686-ed24-4065-be40-dfc9dc2fccaa tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:26.728946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26efb686-ed24-4065-be40-dfc9dc2fccaa tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:26.729385 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26efb686-ed24-4065-be40-dfc9dc2fccaa tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:26.859153 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-26efb686-ed24-4065-be40-dfc9dc2fccaa tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:26.866927 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-26efb686-ed24-4065-be40-dfc9dc2fccaa tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.150337 Jun 10 13:16:26.872721 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 672/1299] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:26 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:27.084860 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6620d662-2a87-409b-b33d-ce46bc08fe20 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:27.085601 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6620d662-2a87-409b-b33d-ce46bc08fe20 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:27.091692 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6620d662-2a87-409b-b33d-ce46bc08fe20 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:27.092139 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6620d662-2a87-409b-b33d-ce46bc08fe20 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:27.092927 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6620d662-2a87-409b-b33d-ce46bc08fe20 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:27.093361 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6620d662-2a87-409b-b33d-ce46bc08fe20 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:27.219716 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6620d662-2a87-409b-b33d-ce46bc08fe20 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:27.221899 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:16:27.228331 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6620d662-2a87-409b-b33d-ce46bc08fe20 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147414 Jun 10 13:16:27.229065 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 673/1300] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:27 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:27.883460 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-795c9e58-1b91-4a66-9065-bbce0a585e80 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:27.884059 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-795c9e58-1b91-4a66-9065-bbce0a585e80 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:27.891127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-795c9e58-1b91-4a66-9065-bbce0a585e80 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:27.891565 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-795c9e58-1b91-4a66-9065-bbce0a585e80 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:27.892388 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-795c9e58-1b91-4a66-9065-bbce0a585e80 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:27.892834 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-795c9e58-1b91-4a66-9065-bbce0a585e80 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:28.023374 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-795c9e58-1b91-4a66-9065-bbce0a585e80 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:28.030769 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-795c9e58-1b91-4a66-9065-bbce0a585e80 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151160 Jun 10 13:16:28.031329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 674/1301] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:27 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:28.245706 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e6353205-65d9-46bf-ab82-1f1074011211 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:28.246309 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e6353205-65d9-46bf-ab82-1f1074011211 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:28.252817 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6353205-65d9-46bf-ab82-1f1074011211 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:28.253266 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6353205-65d9-46bf-ab82-1f1074011211 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:28.254025 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6353205-65d9-46bf-ab82-1f1074011211 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:28.254454 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6353205-65d9-46bf-ab82-1f1074011211 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:28.382956 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e6353205-65d9-46bf-ab82-1f1074011211 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:28.390810 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e6353205-65d9-46bf-ab82-1f1074011211 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148964 Jun 10 13:16:28.391377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 675/1302] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:28 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:29.047517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0eaf8232-5ce1-4a24-8ef8-6b34784b5388 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:29.048156 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0eaf8232-5ce1-4a24-8ef8-6b34784b5388 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:29.054599 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0eaf8232-5ce1-4a24-8ef8-6b34784b5388 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:29.055041 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0eaf8232-5ce1-4a24-8ef8-6b34784b5388 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:29.055825 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0eaf8232-5ce1-4a24-8ef8-6b34784b5388 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:29.056250 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0eaf8232-5ce1-4a24-8ef8-6b34784b5388 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:29.184583 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0eaf8232-5ce1-4a24-8ef8-6b34784b5388 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:29.191994 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0eaf8232-5ce1-4a24-8ef8-6b34784b5388 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148413 Jun 10 13:16:29.192550 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 676/1303] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:29 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:29.408433 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-882526e4-23e9-4ecb-9bb7-7bc9c976e921 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:29.409067 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-882526e4-23e9-4ecb-9bb7-7bc9c976e921 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:29.415021 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-882526e4-23e9-4ecb-9bb7-7bc9c976e921 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:29.415462 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-882526e4-23e9-4ecb-9bb7-7bc9c976e921 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:29.416215 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-882526e4-23e9-4ecb-9bb7-7bc9c976e921 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:29.416643 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-882526e4-23e9-4ecb-9bb7-7bc9c976e921 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:29.544659 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-882526e4-23e9-4ecb-9bb7-7bc9c976e921 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:29.552463 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-882526e4-23e9-4ecb-9bb7-7bc9c976e921 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147875 Jun 10 13:16:29.553054 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 677/1304] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:29 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:30.209440 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ec59a06d-ada6-4f9e-bc1b-40580421bf60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:30.210042 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ec59a06d-ada6-4f9e-bc1b-40580421bf60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:30.217031 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ec59a06d-ada6-4f9e-bc1b-40580421bf60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:30.217474 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ec59a06d-ada6-4f9e-bc1b-40580421bf60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:30.218225 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ec59a06d-ada6-4f9e-bc1b-40580421bf60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:30.218650 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ec59a06d-ada6-4f9e-bc1b-40580421bf60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:30.345279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ec59a06d-ada6-4f9e-bc1b-40580421bf60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:30.352643 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ec59a06d-ada6-4f9e-bc1b-40580421bf60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147065 Jun 10 13:16:30.353225 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 678/1305] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:30 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:30.570256 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-66b31a31-b765-47d1-a90f-3851dd8ce0ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:30.570858 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-66b31a31-b765-47d1-a90f-3851dd8ce0ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:30.577649 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66b31a31-b765-47d1-a90f-3851dd8ce0ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:30.578098 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66b31a31-b765-47d1-a90f-3851dd8ce0ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:30.578902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66b31a31-b765-47d1-a90f-3851dd8ce0ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:30.579329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-66b31a31-b765-47d1-a90f-3851dd8ce0ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:30.705319 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-66b31a31-b765-47d1-a90f-3851dd8ce0ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:30.713035 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-66b31a31-b765-47d1-a90f-3851dd8ce0ff tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147010 Jun 10 13:16:30.713641 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 679/1306] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:30 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:31.371358 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-853ea348-ce68-49f7-bcb5-96d90a783a1b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:31.371966 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-853ea348-ce68-49f7-bcb5-96d90a783a1b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:31.378431 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-853ea348-ce68-49f7-bcb5-96d90a783a1b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:31.378872 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-853ea348-ce68-49f7-bcb5-96d90a783a1b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:31.379619 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-853ea348-ce68-49f7-bcb5-96d90a783a1b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:31.380045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-853ea348-ce68-49f7-bcb5-96d90a783a1b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:31.506388 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-853ea348-ce68-49f7-bcb5-96d90a783a1b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:31.513790 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-853ea348-ce68-49f7-bcb5-96d90a783a1b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147201 Jun 10 13:16:31.514349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 680/1307] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:31 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:31.730724 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d418c245-d7c6-42d4-a0d1-ccbf6553177f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:31.731245 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d418c245-d7c6-42d4-a0d1-ccbf6553177f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:31.737522 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d418c245-d7c6-42d4-a0d1-ccbf6553177f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:31.737963 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d418c245-d7c6-42d4-a0d1-ccbf6553177f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:31.738710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d418c245-d7c6-42d4-a0d1-ccbf6553177f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:31.739137 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d418c245-d7c6-42d4-a0d1-ccbf6553177f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:31.863818 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d418c245-d7c6-42d4-a0d1-ccbf6553177f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:31.872000 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d418c245-d7c6-42d4-a0d1-ccbf6553177f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146089 Jun 10 13:16:31.872590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 681/1308] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:31 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:32.531275 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f6ef37de-1d0b-4bb7-a537-4289610637e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:32.531875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f6ef37de-1d0b-4bb7-a537-4289610637e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:32.538384 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6ef37de-1d0b-4bb7-a537-4289610637e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:32.538825 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6ef37de-1d0b-4bb7-a537-4289610637e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:32.539579 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6ef37de-1d0b-4bb7-a537-4289610637e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:32.540006 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6ef37de-1d0b-4bb7-a537-4289610637e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:32.671730 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f6ef37de-1d0b-4bb7-a537-4289610637e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:32.679168 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f6ef37de-1d0b-4bb7-a537-4289610637e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.151807 Jun 10 13:16:32.679727 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 682/1309] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:32 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:32.888704 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-735b0af6-559f-4eb0-ac0d-caae5d13e4f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:32.889271 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-735b0af6-559f-4eb0-ac0d-caae5d13e4f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:32.895165 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-735b0af6-559f-4eb0-ac0d-caae5d13e4f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:32.895610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-735b0af6-559f-4eb0-ac0d-caae5d13e4f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:32.896369 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-735b0af6-559f-4eb0-ac0d-caae5d13e4f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:32.896821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-735b0af6-559f-4eb0-ac0d-caae5d13e4f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:33.020029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-735b0af6-559f-4eb0-ac0d-caae5d13e4f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:33.028508 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-735b0af6-559f-4eb0-ac0d-caae5d13e4f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144116 Jun 10 13:16:33.029129 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 683/1310] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:32 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:33.696429 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f9026ed5-6afa-4d41-a168-f6c9edee26a8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:33.697097 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f9026ed5-6afa-4d41-a168-f6c9edee26a8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:33.700316 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=79,nova_api:UPDATE=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:16:33.705258 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f9026ed5-6afa-4d41-a168-f6c9edee26a8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:33.705707 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f9026ed5-6afa-4d41-a168-f6c9edee26a8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:33.706469 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f9026ed5-6afa-4d41-a168-f6c9edee26a8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:33.706909 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f9026ed5-6afa-4d41-a168-f6c9edee26a8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:33.839208 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f9026ed5-6afa-4d41-a168-f6c9edee26a8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:33.846795 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f9026ed5-6afa-4d41-a168-f6c9edee26a8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.154226 Jun 10 13:16:33.847363 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 684/1311] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:33 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:34.045468 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-31c1bbd4-6ca6-492b-afa2-d5ade52bfe0d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:34.045984 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-31c1bbd4-6ca6-492b-afa2-d5ade52bfe0d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:34.051969 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-31c1bbd4-6ca6-492b-afa2-d5ade52bfe0d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:34.052412 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-31c1bbd4-6ca6-492b-afa2-d5ade52bfe0d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:34.053198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-31c1bbd4-6ca6-492b-afa2-d5ade52bfe0d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:34.053626 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-31c1bbd4-6ca6-492b-afa2-d5ade52bfe0d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:34.178930 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-31c1bbd4-6ca6-492b-afa2-d5ade52bfe0d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:34.187433 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-31c1bbd4-6ca6-492b-afa2-d5ade52bfe0d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145840 Jun 10 13:16:34.187993 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 685/1312] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:34 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:34.863621 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-468293b1-8168-4aa2-9406-400426c75747 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:34.864276 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-468293b1-8168-4aa2-9406-400426c75747 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:34.870686 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-468293b1-8168-4aa2-9406-400426c75747 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:34.871125 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-468293b1-8168-4aa2-9406-400426c75747 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:34.871871 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-468293b1-8168-4aa2-9406-400426c75747 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:34.872296 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-468293b1-8168-4aa2-9406-400426c75747 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:34.996081 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-468293b1-8168-4aa2-9406-400426c75747 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:35.003688 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-468293b1-8168-4aa2-9406-400426c75747 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.144132 Jun 10 13:16:35.004252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 686/1313] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:34 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:35.208416 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-62f45afc-b108-4e05-a0b4-2be908d21386 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:35.209089 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-62f45afc-b108-4e05-a0b4-2be908d21386 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:35.215806 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62f45afc-b108-4e05-a0b4-2be908d21386 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:35.216253 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62f45afc-b108-4e05-a0b4-2be908d21386 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:35.217047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62f45afc-b108-4e05-a0b4-2be908d21386 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:35.217479 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-62f45afc-b108-4e05-a0b4-2be908d21386 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:35.346127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-62f45afc-b108-4e05-a0b4-2be908d21386 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:35.354022 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-62f45afc-b108-4e05-a0b4-2be908d21386 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.149446 Jun 10 13:16:35.354634 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 687/1314] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:35 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:36.021204 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-daa4e525-ccdc-4272-aece-dec502c11877 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:36.021866 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-daa4e525-ccdc-4272-aece-dec502c11877 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:36.028224 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-daa4e525-ccdc-4272-aece-dec502c11877 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:36.028665 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-daa4e525-ccdc-4272-aece-dec502c11877 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:36.029516 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-daa4e525-ccdc-4272-aece-dec502c11877 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:36.029943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-daa4e525-ccdc-4272-aece-dec502c11877 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:36.156140 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-daa4e525-ccdc-4272-aece-dec502c11877 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:36.163543 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-daa4e525-ccdc-4272-aece-dec502c11877 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146208 Jun 10 13:16:36.164098 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 688/1315] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:36 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:36.371214 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-384fd62e-38ce-40a8-8657-65b91c13a54d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:36.371818 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-384fd62e-38ce-40a8-8657-65b91c13a54d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:36.377796 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-384fd62e-38ce-40a8-8657-65b91c13a54d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:36.378239 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-384fd62e-38ce-40a8-8657-65b91c13a54d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:36.378983 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-384fd62e-38ce-40a8-8657-65b91c13a54d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:36.379415 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-384fd62e-38ce-40a8-8657-65b91c13a54d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:36.505132 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-384fd62e-38ce-40a8-8657-65b91c13a54d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:36.513386 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-384fd62e-38ce-40a8-8657-65b91c13a54d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146053 Jun 10 13:16:36.513949 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 689/1316] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:36 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:37.181090 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d7a5a55c-b1e8-4b58-ab64-be29eedae209 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:37.181784 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d7a5a55c-b1e8-4b58-ab64-be29eedae209 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:37.189005 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d7a5a55c-b1e8-4b58-ab64-be29eedae209 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:37.189448 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d7a5a55c-b1e8-4b58-ab64-be29eedae209 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:37.191081 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d7a5a55c-b1e8-4b58-ab64-be29eedae209 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:37.191661 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d7a5a55c-b1e8-4b58-ab64-be29eedae209 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:37.207221 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:16:37.323057 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d7a5a55c-b1e8-4b58-ab64-be29eedae209 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:37.330963 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d7a5a55c-b1e8-4b58-ab64-be29eedae209 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.153761 Jun 10 13:16:37.331534 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 690/1317] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:37 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:37.530094 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-acf1cf92-2aa8-4406-b23c-82219321622a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:37.530697 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-acf1cf92-2aa8-4406-b23c-82219321622a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:37.536584 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-acf1cf92-2aa8-4406-b23c-82219321622a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:37.537058 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-acf1cf92-2aa8-4406-b23c-82219321622a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:37.537811 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-acf1cf92-2aa8-4406-b23c-82219321622a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:37.538243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-acf1cf92-2aa8-4406-b23c-82219321622a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:37.671182 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-acf1cf92-2aa8-4406-b23c-82219321622a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:37.679009 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-acf1cf92-2aa8-4406-b23c-82219321622a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.152666 Jun 10 13:16:37.679565 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 691/1318] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:37 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:38.347446 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-907343fe-7be4-47d8-aefa-5243bb302093 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:38.347995 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-907343fe-7be4-47d8-aefa-5243bb302093 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:38.355114 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-907343fe-7be4-47d8-aefa-5243bb302093 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:38.355553 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-907343fe-7be4-47d8-aefa-5243bb302093 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:38.356566 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-907343fe-7be4-47d8-aefa-5243bb302093 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:38.357040 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-907343fe-7be4-47d8-aefa-5243bb302093 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:38.482916 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-907343fe-7be4-47d8-aefa-5243bb302093 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:38.490588 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-907343fe-7be4-47d8-aefa-5243bb302093 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147054 Jun 10 13:16:38.491146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 692/1319] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:38 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:38.695572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9a302925-754d-48fc-a831-6e4538631e9c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:38.696200 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9a302925-754d-48fc-a831-6e4538631e9c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:38.702760 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9a302925-754d-48fc-a831-6e4538631e9c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:38.703287 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9a302925-754d-48fc-a831-6e4538631e9c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:38.704521 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9a302925-754d-48fc-a831-6e4538631e9c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:38.704997 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9a302925-754d-48fc-a831-6e4538631e9c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:38.831294 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9a302925-754d-48fc-a831-6e4538631e9c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:38.839756 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9a302925-754d-48fc-a831-6e4538631e9c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148142 Jun 10 13:16:38.840317 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 693/1320] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:38 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:39.507912 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e20b3ebb-e8dd-4ee2-a533-341120bed769 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:39.508549 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e20b3ebb-e8dd-4ee2-a533-341120bed769 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:39.514710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e20b3ebb-e8dd-4ee2-a533-341120bed769 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:39.515156 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e20b3ebb-e8dd-4ee2-a533-341120bed769 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:39.515912 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e20b3ebb-e8dd-4ee2-a533-341120bed769 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:39.516342 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e20b3ebb-e8dd-4ee2-a533-341120bed769 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:39.645379 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e20b3ebb-e8dd-4ee2-a533-341120bed769 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:39.652731 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e20b3ebb-e8dd-4ee2-a533-341120bed769 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148658 Jun 10 13:16:39.653313 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 694/1321] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:39 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:39.857576 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-88d1f7ef-db5e-459d-9dee-a34d2b0b7f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:39.858109 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-88d1f7ef-db5e-459d-9dee-a34d2b0b7f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:39.864307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-88d1f7ef-db5e-459d-9dee-a34d2b0b7f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:39.864759 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-88d1f7ef-db5e-459d-9dee-a34d2b0b7f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:39.865572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-88d1f7ef-db5e-459d-9dee-a34d2b0b7f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:39.866002 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-88d1f7ef-db5e-459d-9dee-a34d2b0b7f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:39.989939 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-88d1f7ef-db5e-459d-9dee-a34d2b0b7f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:39.997861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-88d1f7ef-db5e-459d-9dee-a34d2b0b7f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144827 Jun 10 13:16:39.998433 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 695/1322] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:39 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:40.671194 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-32aab65c-2a23-425e-b79a-b76e90b9727b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:40.671821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-32aab65c-2a23-425e-b79a-b76e90b9727b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:40.678236 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-32aab65c-2a23-425e-b79a-b76e90b9727b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:40.678683 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-32aab65c-2a23-425e-b79a-b76e90b9727b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:40.679446 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-32aab65c-2a23-425e-b79a-b76e90b9727b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:40.679875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-32aab65c-2a23-425e-b79a-b76e90b9727b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:40.809381 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-32aab65c-2a23-425e-b79a-b76e90b9727b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:40.816985 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-32aab65c-2a23-425e-b79a-b76e90b9727b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149797 Jun 10 13:16:40.817645 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 696/1323] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:40 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:41.014783 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1e7327e5-2db5-4d8f-9c98-2ae0eb18a1f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:41.015421 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1e7327e5-2db5-4d8f-9c98-2ae0eb18a1f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:41.021393 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1e7327e5-2db5-4d8f-9c98-2ae0eb18a1f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:41.021839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1e7327e5-2db5-4d8f-9c98-2ae0eb18a1f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:41.022613 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1e7327e5-2db5-4d8f-9c98-2ae0eb18a1f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:41.023047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1e7327e5-2db5-4d8f-9c98-2ae0eb18a1f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:41.149948 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1e7327e5-2db5-4d8f-9c98-2ae0eb18a1f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:41.157906 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1e7327e5-2db5-4d8f-9c98-2ae0eb18a1f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146910 Jun 10 13:16:41.158467 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 697/1324] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:41 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:41.834053 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-17672a04-807b-4348-8801-202963139ac6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:41.834729 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-17672a04-807b-4348-8801-202963139ac6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:41.841802 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-17672a04-807b-4348-8801-202963139ac6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:41.842249 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-17672a04-807b-4348-8801-202963139ac6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:41.843013 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-17672a04-807b-4348-8801-202963139ac6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:41.843445 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-17672a04-807b-4348-8801-202963139ac6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:41.971823 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-17672a04-807b-4348-8801-202963139ac6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:41.979384 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-17672a04-807b-4348-8801-202963139ac6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149259 Jun 10 13:16:41.979957 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 698/1325] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:41 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:42.174991 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2a72af82-0151-4e6a-b2f0-7b48b1d4edfd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:42.175604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2a72af82-0151-4e6a-b2f0-7b48b1d4edfd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:42.181544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2a72af82-0151-4e6a-b2f0-7b48b1d4edfd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:42.182503 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2a72af82-0151-4e6a-b2f0-7b48b1d4edfd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:42.183262 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2a72af82-0151-4e6a-b2f0-7b48b1d4edfd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:42.183700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2a72af82-0151-4e6a-b2f0-7b48b1d4edfd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:42.310856 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2a72af82-0151-4e6a-b2f0-7b48b1d4edfd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:42.318820 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2a72af82-0151-4e6a-b2f0-7b48b1d4edfd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147511 Jun 10 13:16:42.319390 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 699/1326] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:42 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:42.996669 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a0db0c29-3c07-403e-9e20-9b0f5ac2aef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:42.997372 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a0db0c29-3c07-403e-9e20-9b0f5ac2aef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:43.003876 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a0db0c29-3c07-403e-9e20-9b0f5ac2aef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:43.004321 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a0db0c29-3c07-403e-9e20-9b0f5ac2aef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:43.005113 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a0db0c29-3c07-403e-9e20-9b0f5ac2aef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:43.005549 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a0db0c29-3c07-403e-9e20-9b0f5ac2aef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:43.132325 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a0db0c29-3c07-403e-9e20-9b0f5ac2aef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:43.139919 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a0db0c29-3c07-403e-9e20-9b0f5ac2aef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147145 Jun 10 13:16:43.140473 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 700/1327] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:42 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:43.336608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-462111be-47f0-4520-a863-ecc21b4763f8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:43.337144 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-462111be-47f0-4520-a863-ecc21b4763f8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:43.343811 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-462111be-47f0-4520-a863-ecc21b4763f8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:43.344249 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-462111be-47f0-4520-a863-ecc21b4763f8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:43.345037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-462111be-47f0-4520-a863-ecc21b4763f8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:43.345468 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-462111be-47f0-4520-a863-ecc21b4763f8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:43.471489 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-462111be-47f0-4520-a863-ecc21b4763f8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:43.479231 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-462111be-47f0-4520-a863-ecc21b4763f8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146593 Jun 10 13:16:43.479788 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 701/1328] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:43 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:44.156851 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e17e7a86-a001-4455-8980-9322e0d8a1d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:44.157457 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e17e7a86-a001-4455-8980-9322e0d8a1d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:44.164022 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e17e7a86-a001-4455-8980-9322e0d8a1d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:44.164463 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e17e7a86-a001-4455-8980-9322e0d8a1d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:44.165239 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e17e7a86-a001-4455-8980-9322e0d8a1d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:44.165667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e17e7a86-a001-4455-8980-9322e0d8a1d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:44.293735 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e17e7a86-a001-4455-8980-9322e0d8a1d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:44.301105 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e17e7a86-a001-4455-8980-9322e0d8a1d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148160 Jun 10 13:16:44.301664 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 702/1329] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:44 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:44.495422 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-976d2f78-4b45-4492-885e-85cf67f7901f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:44.496026 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-976d2f78-4b45-4492-885e-85cf67f7901f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:44.501921 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-976d2f78-4b45-4492-885e-85cf67f7901f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:44.502365 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-976d2f78-4b45-4492-885e-85cf67f7901f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:44.503113 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-976d2f78-4b45-4492-885e-85cf67f7901f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:44.503540 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-976d2f78-4b45-4492-885e-85cf67f7901f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:44.632151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-976d2f78-4b45-4492-885e-85cf67f7901f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:44.640270 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-976d2f78-4b45-4492-885e-85cf67f7901f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148562 Jun 10 13:16:44.640896 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 703/1330] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:44 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:45.318244 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8f3c26c4-f477-45f5-9732-076bb95256c3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:45.318938 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8f3c26c4-f477-45f5-9732-076bb95256c3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:45.327578 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8f3c26c4-f477-45f5-9732-076bb95256c3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:45.328028 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8f3c26c4-f477-45f5-9732-076bb95256c3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:45.328845 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8f3c26c4-f477-45f5-9732-076bb95256c3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:45.329307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8f3c26c4-f477-45f5-9732-076bb95256c3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:45.463636 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8f3c26c4-f477-45f5-9732-076bb95256c3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:45.471186 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8f3c26c4-f477-45f5-9732-076bb95256c3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.157153 Jun 10 13:16:45.471747 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 704/1331] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:45 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:45.656842 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b4f3c2cc-e28b-46f8-a60f-7713829cf5cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:45.657464 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b4f3c2cc-e28b-46f8-a60f-7713829cf5cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:45.663424 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b4f3c2cc-e28b-46f8-a60f-7713829cf5cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:45.663869 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b4f3c2cc-e28b-46f8-a60f-7713829cf5cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:45.664626 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b4f3c2cc-e28b-46f8-a60f-7713829cf5cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:45.665087 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b4f3c2cc-e28b-46f8-a60f-7713829cf5cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:45.799237 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b4f3c2cc-e28b-46f8-a60f-7713829cf5cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:45.808139 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b4f3c2cc-e28b-46f8-a60f-7713829cf5cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.155136 Jun 10 13:16:45.808693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 705/1332] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:45 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:46.488942 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f2747530-3792-4f58-a445-a3f50b58db81 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:46.489555 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f2747530-3792-4f58-a445-a3f50b58db81 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:46.496029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f2747530-3792-4f58-a445-a3f50b58db81 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:46.496472 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f2747530-3792-4f58-a445-a3f50b58db81 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:46.497252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f2747530-3792-4f58-a445-a3f50b58db81 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:46.497683 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f2747530-3792-4f58-a445-a3f50b58db81 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:46.630232 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f2747530-3792-4f58-a445-a3f50b58db81 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:46.638498 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f2747530-3792-4f58-a445-a3f50b58db81 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.153367 Jun 10 13:16:46.639074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 706/1333] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:46 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:46.826900 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7b7eacdc-b6dd-495c-ba2a-edca4f755a82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:46.827427 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7b7eacdc-b6dd-495c-ba2a-edca4f755a82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:46.833658 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7b7eacdc-b6dd-495c-ba2a-edca4f755a82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:46.834099 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7b7eacdc-b6dd-495c-ba2a-edca4f755a82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:46.843979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7b7eacdc-b6dd-495c-ba2a-edca4f755a82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:46.843979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7b7eacdc-b6dd-495c-ba2a-edca4f755a82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:46.843979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:16:46.959916 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7b7eacdc-b6dd-495c-ba2a-edca4f755a82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:46.967823 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7b7eacdc-b6dd-495c-ba2a-edca4f755a82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145953 Jun 10 13:16:46.968391 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 707/1334] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:46 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:47.655596 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9b343947-933e-434b-a016-daefa6e73ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:47.656211 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9b343947-933e-434b-a016-daefa6e73ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:47.665119 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9b343947-933e-434b-a016-daefa6e73ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:47.665565 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9b343947-933e-434b-a016-daefa6e73ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:47.666898 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9b343947-933e-434b-a016-daefa6e73ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:47.667334 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9b343947-933e-434b-a016-daefa6e73ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:47.800046 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9b343947-933e-434b-a016-daefa6e73ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:47.807476 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9b343947-933e-434b-a016-daefa6e73ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.155903 Jun 10 13:16:47.808041 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 708/1335] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:47 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:47.985218 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c06c0762-cfdf-4091-bcfa-331a50b01c50 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:47.985821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c06c0762-cfdf-4091-bcfa-331a50b01c50 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:47.991752 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c06c0762-cfdf-4091-bcfa-331a50b01c50 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:47.992192 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c06c0762-cfdf-4091-bcfa-331a50b01c50 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:47.992964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c06c0762-cfdf-4091-bcfa-331a50b01c50 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:47.993393 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c06c0762-cfdf-4091-bcfa-331a50b01c50 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:48.116884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c06c0762-cfdf-4091-bcfa-331a50b01c50 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:48.125146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c06c0762-cfdf-4091-bcfa-331a50b01c50 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144079 Jun 10 13:16:48.125710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 709/1336] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:47 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:48.824220 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-63d611e2-e022-4afc-9236-cc25a23f96b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:48.824933 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-63d611e2-e022-4afc-9236-cc25a23f96b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:48.831708 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-63d611e2-e022-4afc-9236-cc25a23f96b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:48.832155 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-63d611e2-e022-4afc-9236-cc25a23f96b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:48.832956 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-63d611e2-e022-4afc-9236-cc25a23f96b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:48.833388 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-63d611e2-e022-4afc-9236-cc25a23f96b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:48.961197 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-63d611e2-e022-4afc-9236-cc25a23f96b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:48.969321 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-63d611e2-e022-4afc-9236-cc25a23f96b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149013 Jun 10 13:16:48.969885 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 710/1337] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:48 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:49.142096 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-75134bef-c8b6-4ff1-ad26-990e6f7d35ad tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:49.142790 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-75134bef-c8b6-4ff1-ad26-990e6f7d35ad tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:49.148677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-75134bef-c8b6-4ff1-ad26-990e6f7d35ad tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:49.149388 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-75134bef-c8b6-4ff1-ad26-990e6f7d35ad tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:49.150014 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-75134bef-c8b6-4ff1-ad26-990e6f7d35ad tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:49.150441 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-75134bef-c8b6-4ff1-ad26-990e6f7d35ad tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:49.283257 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-75134bef-c8b6-4ff1-ad26-990e6f7d35ad tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:49.291039 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-75134bef-c8b6-4ff1-ad26-990e6f7d35ad tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.152703 Jun 10 13:16:49.291596 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 711/1338] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:49 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:49.986257 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e758fb00-3a0e-4e12-ada6-d4bd995b3ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:49.995503 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e758fb00-3a0e-4e12-ada6-d4bd995b3ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:49.995503 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e758fb00-3a0e-4e12-ada6-d4bd995b3ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:49.995503 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e758fb00-3a0e-4e12-ada6-d4bd995b3ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:49.995503 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e758fb00-3a0e-4e12-ada6-d4bd995b3ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:49.995907 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e758fb00-3a0e-4e12-ada6-d4bd995b3ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:50.121740 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e758fb00-3a0e-4e12-ada6-d4bd995b3ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:50.129191 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e758fb00-3a0e-4e12-ada6-d4bd995b3ab6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147095 Jun 10 13:16:50.129746 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 712/1339] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:49 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:50.308721 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ffe5c6dd-c864-449c-8fdb-273571229f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:50.309284 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ffe5c6dd-c864-449c-8fdb-273571229f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:50.315089 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ffe5c6dd-c864-449c-8fdb-273571229f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:50.315534 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ffe5c6dd-c864-449c-8fdb-273571229f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:50.316290 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ffe5c6dd-c864-449c-8fdb-273571229f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:50.316723 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ffe5c6dd-c864-449c-8fdb-273571229f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:50.445412 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ffe5c6dd-c864-449c-8fdb-273571229f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:50.453842 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ffe5c6dd-c864-449c-8fdb-273571229f61 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.149401 Jun 10 13:16:50.454406 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 713/1340] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:50 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:51.146560 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-72f5cdce-fb7c-484a-b15e-848636e7bcd0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:51.147178 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-72f5cdce-fb7c-484a-b15e-848636e7bcd0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:51.153652 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-72f5cdce-fb7c-484a-b15e-848636e7bcd0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:51.154098 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-72f5cdce-fb7c-484a-b15e-848636e7bcd0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:51.154852 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-72f5cdce-fb7c-484a-b15e-848636e7bcd0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:51.155283 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-72f5cdce-fb7c-484a-b15e-848636e7bcd0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:51.282458 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-72f5cdce-fb7c-484a-b15e-848636e7bcd0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:51.289921 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-72f5cdce-fb7c-484a-b15e-848636e7bcd0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147152 Jun 10 13:16:51.290479 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 714/1341] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:51 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:51.470762 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-52da1b95-9477-4653-a83d-17bc8f1eb981 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:51.471369 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-52da1b95-9477-4653-a83d-17bc8f1eb981 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:51.477327 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52da1b95-9477-4653-a83d-17bc8f1eb981 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:51.477775 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52da1b95-9477-4653-a83d-17bc8f1eb981 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:51.478527 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52da1b95-9477-4653-a83d-17bc8f1eb981 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:51.478959 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-52da1b95-9477-4653-a83d-17bc8f1eb981 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:51.605944 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-52da1b95-9477-4653-a83d-17bc8f1eb981 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:51.614256 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-52da1b95-9477-4653-a83d-17bc8f1eb981 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147337 Jun 10 13:16:51.614822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 715/1342] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:51 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:52.306857 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d95ab94e-ff52-41f9-b138-ab24e04c9a2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:52.308091 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d95ab94e-ff52-41f9-b138-ab24e04c9a2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:52.314567 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d95ab94e-ff52-41f9-b138-ab24e04c9a2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:52.315010 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d95ab94e-ff52-41f9-b138-ab24e04c9a2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:52.315767 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d95ab94e-ff52-41f9-b138-ab24e04c9a2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:52.316196 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d95ab94e-ff52-41f9-b138-ab24e04c9a2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:52.441495 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d95ab94e-ff52-41f9-b138-ab24e04c9a2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:52.448950 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d95ab94e-ff52-41f9-b138-ab24e04c9a2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145920 Jun 10 13:16:52.449513 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 716/1343] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:52 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:52.631804 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ec2c0c46-82b7-4bc6-8506-786755a7ae69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:52.632406 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ec2c0c46-82b7-4bc6-8506-786755a7ae69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:52.638288 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ec2c0c46-82b7-4bc6-8506-786755a7ae69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:52.638738 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ec2c0c46-82b7-4bc6-8506-786755a7ae69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:52.639495 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ec2c0c46-82b7-4bc6-8506-786755a7ae69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:52.639929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ec2c0c46-82b7-4bc6-8506-786755a7ae69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:52.763124 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ec2c0c46-82b7-4bc6-8506-786755a7ae69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:52.770873 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ec2c0c46-82b7-4bc6-8506-786755a7ae69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.142691 Jun 10 13:16:52.771433 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 717/1344] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:52 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:53.465833 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cbeee7b0-0c4f-43ad-8db5-aaecd6c8b9c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:53.466471 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cbeee7b0-0c4f-43ad-8db5-aaecd6c8b9c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:53.473039 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbeee7b0-0c4f-43ad-8db5-aaecd6c8b9c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:53.473486 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbeee7b0-0c4f-43ad-8db5-aaecd6c8b9c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:53.474243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbeee7b0-0c4f-43ad-8db5-aaecd6c8b9c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:53.474674 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cbeee7b0-0c4f-43ad-8db5-aaecd6c8b9c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:53.602428 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cbeee7b0-0c4f-43ad-8db5-aaecd6c8b9c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:53.610336 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cbeee7b0-0c4f-43ad-8db5-aaecd6c8b9c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148384 Jun 10 13:16:53.610894 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 718/1345] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:53 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:53.787480 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5911afea-6d59-43fa-ad13-2787d43ec9cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:53.788129 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5911afea-6d59-43fa-ad13-2787d43ec9cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:53.794751 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5911afea-6d59-43fa-ad13-2787d43ec9cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:53.795203 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5911afea-6d59-43fa-ad13-2787d43ec9cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:53.795962 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5911afea-6d59-43fa-ad13-2787d43ec9cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:53.796398 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5911afea-6d59-43fa-ad13-2787d43ec9cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:53.922302 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5911afea-6d59-43fa-ad13-2787d43ec9cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:53.930250 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5911afea-6d59-43fa-ad13-2787d43ec9cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146672 Jun 10 13:16:53.930806 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 719/1346] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:53 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:54.627294 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-65032ee5-90a7-4809-b531-70e0846323c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:54.627878 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-65032ee5-90a7-4809-b531-70e0846323c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:54.634388 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-65032ee5-90a7-4809-b531-70e0846323c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:54.634826 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-65032ee5-90a7-4809-b531-70e0846323c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:54.636118 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-65032ee5-90a7-4809-b531-70e0846323c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:54.636541 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-65032ee5-90a7-4809-b531-70e0846323c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:54.765864 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-65032ee5-90a7-4809-b531-70e0846323c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:54.773265 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-65032ee5-90a7-4809-b531-70e0846323c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.149877 Jun 10 13:16:54.773848 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 720/1347] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:54 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:54.947721 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1a54ddbb-422c-4b4f-b126-28bb6e09efbb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:54.948949 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1a54ddbb-422c-4b4f-b126-28bb6e09efbb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:54.954991 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1a54ddbb-422c-4b4f-b126-28bb6e09efbb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:54.955436 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1a54ddbb-422c-4b4f-b126-28bb6e09efbb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:54.956193 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1a54ddbb-422c-4b4f-b126-28bb6e09efbb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:54.956624 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1a54ddbb-422c-4b4f-b126-28bb6e09efbb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:55.081547 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1a54ddbb-422c-4b4f-b126-28bb6e09efbb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:55.089444 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1a54ddbb-422c-4b4f-b126-28bb6e09efbb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145539 Jun 10 13:16:55.090011 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 721/1348] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:54 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:55.791287 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-faca69c2-c751-48af-9eaa-f51d78717c8a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:55.792586 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-faca69c2-c751-48af-9eaa-f51d78717c8a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:55.799056 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-faca69c2-c751-48af-9eaa-f51d78717c8a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:55.799496 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-faca69c2-c751-48af-9eaa-f51d78717c8a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:55.800241 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-faca69c2-c751-48af-9eaa-f51d78717c8a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:55.800668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-faca69c2-c751-48af-9eaa-f51d78717c8a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:55.926099 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-faca69c2-c751-48af-9eaa-f51d78717c8a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:55.934735 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-faca69c2-c751-48af-9eaa-f51d78717c8a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147452 Jun 10 13:16:55.935315 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 722/1349] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:55 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:56.106509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-aa69c04b-b2cb-482b-a4cf-aaeb0f28ff0c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:56.106766 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-aa69c04b-b2cb-482b-a4cf-aaeb0f28ff0c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:56.113174 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-aa69c04b-b2cb-482b-a4cf-aaeb0f28ff0c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:56.113619 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-aa69c04b-b2cb-482b-a4cf-aaeb0f28ff0c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:56.114408 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-aa69c04b-b2cb-482b-a4cf-aaeb0f28ff0c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:56.114840 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-aa69c04b-b2cb-482b-a4cf-aaeb0f28ff0c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:56.258332 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-aa69c04b-b2cb-482b-a4cf-aaeb0f28ff0c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:56.261038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:16:56.267293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-aa69c04b-b2cb-482b-a4cf-aaeb0f28ff0c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.164855 Jun 10 13:16:56.273918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 723/1350] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:56 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:56.951721 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-46514071-a965-4e22-9387-520b538d21dd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:56.952381 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-46514071-a965-4e22-9387-520b538d21dd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:56.959178 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46514071-a965-4e22-9387-520b538d21dd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:56.959623 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46514071-a965-4e22-9387-520b538d21dd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:56.960381 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46514071-a965-4e22-9387-520b538d21dd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:56.960836 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46514071-a965-4e22-9387-520b538d21dd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:57.087622 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-46514071-a965-4e22-9387-520b538d21dd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:57.095575 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-46514071-a965-4e22-9387-520b538d21dd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147801 Jun 10 13:16:57.096146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 724/1351] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:56 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:16:57.237268 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:16:57.237502 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Fri Jun 10 13:16:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:16:57.289673 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ef80233d-ded2-4e59-9c4b-1ea81bee54a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:57.290310 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ef80233d-ded2-4e59-9c4b-1ea81bee54a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:57.297034 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef80233d-ded2-4e59-9c4b-1ea81bee54a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:57.297483 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef80233d-ded2-4e59-9c4b-1ea81bee54a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:57.298237 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef80233d-ded2-4e59-9c4b-1ea81bee54a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:57.298671 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef80233d-ded2-4e59-9c4b-1ea81bee54a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:57.436929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ef80233d-ded2-4e59-9c4b-1ea81bee54a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:57.444826 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ef80233d-ded2-4e59-9c4b-1ea81bee54a5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.158911 Jun 10 13:16:57.445393 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 725/1352] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:57 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:58.114001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3759f289-a9ca-427a-92c1-fc379d01064c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:58.114611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3759f289-a9ca-427a-92c1-fc379d01064c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:58.121122 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3759f289-a9ca-427a-92c1-fc379d01064c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:58.121568 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3759f289-a9ca-427a-92c1-fc379d01064c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:58.122349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3759f289-a9ca-427a-92c1-fc379d01064c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:58.122780 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3759f289-a9ca-427a-92c1-fc379d01064c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:58.247364 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3759f289-a9ca-427a-92c1-fc379d01064c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:58.254787 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3759f289-a9ca-427a-92c1-fc379d01064c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.144614 Jun 10 13:16:58.255345 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 726/1353] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:58 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:58.461762 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e6d829c0-894d-4de1-81f3-3fceeacafd7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:58.462396 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e6d829c0-894d-4de1-81f3-3fceeacafd7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:58.468301 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6d829c0-894d-4de1-81f3-3fceeacafd7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:58.468739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6d829c0-894d-4de1-81f3-3fceeacafd7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:58.469516 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6d829c0-894d-4de1-81f3-3fceeacafd7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:58.469941 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e6d829c0-894d-4de1-81f3-3fceeacafd7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:58.598022 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e6d829c0-894d-4de1-81f3-3fceeacafd7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:58.606551 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e6d829c0-894d-4de1-81f3-3fceeacafd7b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148563 Jun 10 13:16:58.607628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 727/1354] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:58 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:59.271714 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f50dc8eb-3072-4e9e-8788-487bd04e7b96 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:59.272385 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f50dc8eb-3072-4e9e-8788-487bd04e7b96 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:59.279422 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f50dc8eb-3072-4e9e-8788-487bd04e7b96 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:59.279857 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f50dc8eb-3072-4e9e-8788-487bd04e7b96 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:59.280602 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f50dc8eb-3072-4e9e-8788-487bd04e7b96 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:59.281063 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f50dc8eb-3072-4e9e-8788-487bd04e7b96 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:59.405998 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f50dc8eb-3072-4e9e-8788-487bd04e7b96 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:59.413375 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f50dc8eb-3072-4e9e-8788-487bd04e7b96 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.145487 Jun 10 13:16:59.413929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 728/1355] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:59 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:16:59.623950 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6b405270-6bc7-4b8a-865a-7a8d7d3e2e8d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:16:59.624492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6b405270-6bc7-4b8a-865a-7a8d7d3e2e8d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:16:59.630497 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6b405270-6bc7-4b8a-865a-7a8d7d3e2e8d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:59.630947 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6b405270-6bc7-4b8a-865a-7a8d7d3e2e8d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:59.631709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6b405270-6bc7-4b8a-865a-7a8d7d3e2e8d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:16:59.632148 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6b405270-6bc7-4b8a-865a-7a8d7d3e2e8d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:16:59.761652 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6b405270-6bc7-4b8a-865a-7a8d7d3e2e8d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:16:59.769501 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6b405270-6bc7-4b8a-865a-7a8d7d3e2e8d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.149988 Jun 10 13:16:59.770071 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 729/1356] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:16:59 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:00.430356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-12b205df-14cb-43f1-b4e0-fb430b90a114 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:00.430933 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-12b205df-14cb-43f1-b4e0-fb430b90a114 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:00.437330 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12b205df-14cb-43f1-b4e0-fb430b90a114 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:00.437770 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12b205df-14cb-43f1-b4e0-fb430b90a114 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:00.438515 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12b205df-14cb-43f1-b4e0-fb430b90a114 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:00.438937 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12b205df-14cb-43f1-b4e0-fb430b90a114 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:00.565750 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-12b205df-14cb-43f1-b4e0-fb430b90a114 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:00.573562 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-12b205df-14cb-43f1-b4e0-fb430b90a114 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.146549 Jun 10 13:17:00.574146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 730/1357] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:00 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:00.785985 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-26a96797-9185-4030-b7e4-78595b7d1d2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:00.786585 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-26a96797-9185-4030-b7e4-78595b7d1d2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:00.792361 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26a96797-9185-4030-b7e4-78595b7d1d2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:00.792824 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26a96797-9185-4030-b7e4-78595b7d1d2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:00.793569 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26a96797-9185-4030-b7e4-78595b7d1d2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:00.794000 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26a96797-9185-4030-b7e4-78595b7d1d2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:00.918262 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-26a96797-9185-4030-b7e4-78595b7d1d2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:00.926128 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-26a96797-9185-4030-b7e4-78595b7d1d2f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143750 Jun 10 13:17:00.926686 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 731/1358] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:00 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:01.589968 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-dd1742cd-0f39-4020-8644-f1571356f0d1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:01.590633 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-dd1742cd-0f39-4020-8644-f1571356f0d1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:01.597045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dd1742cd-0f39-4020-8644-f1571356f0d1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:01.597492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dd1742cd-0f39-4020-8644-f1571356f0d1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:01.598247 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dd1742cd-0f39-4020-8644-f1571356f0d1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:01.598685 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dd1742cd-0f39-4020-8644-f1571356f0d1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:01.726748 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-dd1742cd-0f39-4020-8644-f1571356f0d1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:01.734440 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-dd1742cd-0f39-4020-8644-f1571356f0d1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.148182 Jun 10 13:17:01.734994 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 732/1359] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:01 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:01.942918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-de848948-0ddb-4bfb-8b4e-7a7fd77a7002 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:01.943533 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-de848948-0ddb-4bfb-8b4e-7a7fd77a7002 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:01.949548 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-de848948-0ddb-4bfb-8b4e-7a7fd77a7002 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:01.949990 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-de848948-0ddb-4bfb-8b4e-7a7fd77a7002 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:01.950746 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-de848948-0ddb-4bfb-8b4e-7a7fd77a7002 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:01.951177 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-de848948-0ddb-4bfb-8b4e-7a7fd77a7002 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:02.075384 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-de848948-0ddb-4bfb-8b4e-7a7fd77a7002 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:02.083252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-de848948-0ddb-4bfb-8b4e-7a7fd77a7002 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144199 Jun 10 13:17:02.083816 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 733/1360] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:01 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:02.751437 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9e351598-ed43-4897-bab9-269fdd8d1aea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:02.752069 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9e351598-ed43-4897-bab9-269fdd8d1aea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:02.754405 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:17:02.759151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9e351598-ed43-4897-bab9-269fdd8d1aea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:02.759597 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9e351598-ed43-4897-bab9-269fdd8d1aea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:02.760891 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9e351598-ed43-4897-bab9-269fdd8d1aea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:02.761326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9e351598-ed43-4897-bab9-269fdd8d1aea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:02.887660 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9e351598-ed43-4897-bab9-269fdd8d1aea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:02.895111 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9e351598-ed43-4897-bab9-269fdd8d1aea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.147543 Jun 10 13:17:02.895674 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 734/1361] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:02 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:03.099268 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a6882cc1-40d2-45ae-85d7-2462e7846799 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:03.099877 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a6882cc1-40d2-45ae-85d7-2462e7846799 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:03.106488 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a6882cc1-40d2-45ae-85d7-2462e7846799 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:03.106937 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a6882cc1-40d2-45ae-85d7-2462e7846799 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:03.107693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a6882cc1-40d2-45ae-85d7-2462e7846799 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:03.108124 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a6882cc1-40d2-45ae-85d7-2462e7846799 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:03.238956 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a6882cc1-40d2-45ae-85d7-2462e7846799 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:03.246799 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a6882cc1-40d2-45ae-85d7-2462e7846799 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.151336 Jun 10 13:17:03.247359 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 735/1362] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:03 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:03.912113 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-83c8ebff-92da-4694-9336-d77f5f7808f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:03.912826 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-83c8ebff-92da-4694-9336-d77f5f7808f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:03.919211 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-83c8ebff-92da-4694-9336-d77f5f7808f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:03.919652 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-83c8ebff-92da-4694-9336-d77f5f7808f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:03.920406 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-83c8ebff-92da-4694-9336-d77f5f7808f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:03.920863 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-83c8ebff-92da-4694-9336-d77f5f7808f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:04.052348 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-83c8ebff-92da-4694-9336-d77f5f7808f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:04.060389 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-83c8ebff-92da-4694-9336-d77f5f7808f9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.152174 Jun 10 13:17:04.060985 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 736/1363] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:03 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:04.263412 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6a8a3bcd-11b6-400e-b158-5e7b2598ca82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:04.264020 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6a8a3bcd-11b6-400e-b158-5e7b2598ca82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:04.269965 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6a8a3bcd-11b6-400e-b158-5e7b2598ca82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:04.270416 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6a8a3bcd-11b6-400e-b158-5e7b2598ca82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:04.271177 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6a8a3bcd-11b6-400e-b158-5e7b2598ca82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:04.271606 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6a8a3bcd-11b6-400e-b158-5e7b2598ca82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:04.728596 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6a8a3bcd-11b6-400e-b158-5e7b2598ca82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:04.737610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6a8a3bcd-11b6-400e-b158-5e7b2598ca82 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.478103 Jun 10 13:17:04.739381 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 737/1364] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:04 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:05.364499 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-46b6e443-7885-4599-96a1-f4c6451cacd7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:05.365238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-46b6e443-7885-4599-96a1-f4c6451cacd7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:05.372175 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46b6e443-7885-4599-96a1-f4c6451cacd7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:05.372613 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46b6e443-7885-4599-96a1-f4c6451cacd7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:05.373418 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46b6e443-7885-4599-96a1-f4c6451cacd7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:05.373847 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46b6e443-7885-4599-96a1-f4c6451cacd7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:05.402220 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-46b6e443-7885-4599-96a1-f4c6451cacd7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Going to try to terminate instance {{(pid=118093) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:17:05.513100 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:17:05.691107 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-46b6e443-7885-4599-96a1-f4c6451cacd7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "DELETE /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 204 len: 0 microversion: 2.1 time: 0.330162 Jun 10 13:17:05.691699 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 738/1365] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:17:05 2022] DELETE /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jun 10 13:17:05.745633 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-c1563237-cceb-4442-bb11-f9372ed02221 req-003217a3-e9a9-464f-8eb4-4d3bc3ceb36c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3baaefc5-1a36-4fa6-88d9-1764e5631a32", "tag": "72eeacbd-3288-4d28-90e4-93e208b2d17d"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:17:05.757399 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-c1563237-cceb-4442-bb11-f9372ed02221 req-003217a3-e9a9-464f-8eb4-4d3bc3ceb36c service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:05.757847 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-c1563237-cceb-4442-bb11-f9372ed02221 req-003217a3-e9a9-464f-8eb4-4d3bc3ceb36c service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:05.769694 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-c1563237-cceb-4442-bb11-f9372ed02221 req-003217a3-e9a9-464f-8eb4-4d3bc3ceb36c service nova] Creating event network-changed:72eeacbd-3288-4d28-90e4-93e208b2d17d for instance 3baaefc5-1a36-4fa6-88d9-1764e5631a32 on hv-cinder-maste Jun 10 13:17:05.774412 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-c1563237-cceb-4442-bb11-f9372ed02221 req-003217a3-e9a9-464f-8eb4-4d3bc3ceb36c service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.081455 Jun 10 13:17:05.775004 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 739/1366] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:17:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:17:05.782171 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7a17195e-af5c-4691-8d1b-03b11580af3c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:05.782853 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7a17195e-af5c-4691-8d1b-03b11580af3c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:05.788922 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7a17195e-af5c-4691-8d1b-03b11580af3c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:05.789360 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7a17195e-af5c-4691-8d1b-03b11580af3c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:05.790107 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7a17195e-af5c-4691-8d1b-03b11580af3c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:05.790534 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7a17195e-af5c-4691-8d1b-03b11580af3c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:06.014258 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7a17195e-af5c-4691-8d1b-03b11580af3c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:06.022401 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7a17195e-af5c-4691-8d1b-03b11580af3c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1570 microversion: 2.1 time: 0.244839 Jun 10 13:17:06.022921 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 740/1367] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:05 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1570 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:17:06.036039 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b7534cea-15af-427c-8a20-3b101085e1e2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:06.036647 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b7534cea-15af-427c-8a20-3b101085e1e2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:06.042671 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b7534cea-15af-427c-8a20-3b101085e1e2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:06.044094 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b7534cea-15af-427c-8a20-3b101085e1e2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:06.044094 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b7534cea-15af-427c-8a20-3b101085e1e2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:06.044316 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b7534cea-15af-427c-8a20-3b101085e1e2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:06.247191 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b7534cea-15af-427c-8a20-3b101085e1e2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:06.255814 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b7534cea-15af-427c-8a20-3b101085e1e2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.231884 Jun 10 13:17:06.265190 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 741/1368] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:06 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:17:07.039023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-14c1208f-1ce5-4231-8c79-59e64d312ed4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:07.039705 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-14c1208f-1ce5-4231-8c79-59e64d312ed4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:07.046658 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14c1208f-1ce5-4231-8c79-59e64d312ed4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:07.047098 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14c1208f-1ce5-4231-8c79-59e64d312ed4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:07.047844 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14c1208f-1ce5-4231-8c79-59e64d312ed4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:07.048269 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14c1208f-1ce5-4231-8c79-59e64d312ed4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:07.176120 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-14c1208f-1ce5-4231-8c79-59e64d312ed4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:07.183808 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-14c1208f-1ce5-4231-8c79-59e64d312ed4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.148586 Jun 10 13:17:07.184372 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 742/1369] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:07 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:07.273526 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f8e48663-b603-4612-9b9a-de9971b4454c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:07.274318 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f8e48663-b603-4612-9b9a-de9971b4454c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:07.281248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f8e48663-b603-4612-9b9a-de9971b4454c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:07.281698 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f8e48663-b603-4612-9b9a-de9971b4454c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:07.282467 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f8e48663-b603-4612-9b9a-de9971b4454c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:07.282902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f8e48663-b603-4612-9b9a-de9971b4454c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:07.414139 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f8e48663-b603-4612-9b9a-de9971b4454c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:07.422124 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f8e48663-b603-4612-9b9a-de9971b4454c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.152495 Jun 10 13:17:07.422688 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 743/1370] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:07 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:08.200790 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3c6accf5-1e3c-469a-a080-ff5d29a4ba75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:08.201402 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3c6accf5-1e3c-469a-a080-ff5d29a4ba75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:08.207889 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c6accf5-1e3c-469a-a080-ff5d29a4ba75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:08.208333 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c6accf5-1e3c-469a-a080-ff5d29a4ba75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:08.209118 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c6accf5-1e3c-469a-a080-ff5d29a4ba75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:08.209552 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c6accf5-1e3c-469a-a080-ff5d29a4ba75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:08.336370 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3c6accf5-1e3c-469a-a080-ff5d29a4ba75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:08.343893 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3c6accf5-1e3c-469a-a080-ff5d29a4ba75 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.147006 Jun 10 13:17:08.344450 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 744/1371] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:08 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:08.438081 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3b281bb9-ce05-4f17-baa9-5c0f9209c259 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:08.438614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3b281bb9-ce05-4f17-baa9-5c0f9209c259 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:08.444400 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3b281bb9-ce05-4f17-baa9-5c0f9209c259 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:08.444878 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3b281bb9-ce05-4f17-baa9-5c0f9209c259 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:08.445636 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3b281bb9-ce05-4f17-baa9-5c0f9209c259 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:08.446066 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3b281bb9-ce05-4f17-baa9-5c0f9209c259 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:08.578721 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3b281bb9-ce05-4f17-baa9-5c0f9209c259 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:08.586738 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3b281bb9-ce05-4f17-baa9-5c0f9209c259 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.152261 Jun 10 13:17:08.587306 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 745/1372] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:08 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:09.361700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c33d94be-77b4-4bfb-aa05-9850ec31dc0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:09.362299 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c33d94be-77b4-4bfb-aa05-9850ec31dc0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:09.368473 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c33d94be-77b4-4bfb-aa05-9850ec31dc0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:09.368950 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c33d94be-77b4-4bfb-aa05-9850ec31dc0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:09.369696 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c33d94be-77b4-4bfb-aa05-9850ec31dc0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:09.370123 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c33d94be-77b4-4bfb-aa05-9850ec31dc0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:09.494991 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c33d94be-77b4-4bfb-aa05-9850ec31dc0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:09.502930 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c33d94be-77b4-4bfb-aa05-9850ec31dc0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145291 Jun 10 13:17:09.503483 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 746/1373] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:09 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:09.602960 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-39427ac3-0d65-44dd-b1e7-860092dd7f88 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:09.603483 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-39427ac3-0d65-44dd-b1e7-860092dd7f88 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:09.610076 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-39427ac3-0d65-44dd-b1e7-860092dd7f88 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:09.610513 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-39427ac3-0d65-44dd-b1e7-860092dd7f88 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:09.611259 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-39427ac3-0d65-44dd-b1e7-860092dd7f88 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:09.611683 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-39427ac3-0d65-44dd-b1e7-860092dd7f88 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:09.735205 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-39427ac3-0d65-44dd-b1e7-860092dd7f88 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:09.743022 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-39427ac3-0d65-44dd-b1e7-860092dd7f88 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143724 Jun 10 13:17:09.743579 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 747/1374] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:09 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:10.520293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-55617468-b6d5-4790-88bc-98b1474cd84f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:10.520923 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-55617468-b6d5-4790-88bc-98b1474cd84f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:10.527345 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-55617468-b6d5-4790-88bc-98b1474cd84f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:10.527814 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-55617468-b6d5-4790-88bc-98b1474cd84f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:10.528556 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-55617468-b6d5-4790-88bc-98b1474cd84f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:10.529044 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-55617468-b6d5-4790-88bc-98b1474cd84f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:10.657563 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-55617468-b6d5-4790-88bc-98b1474cd84f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:10.664964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-55617468-b6d5-4790-88bc-98b1474cd84f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.148437 Jun 10 13:17:10.665528 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 748/1375] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:10 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:10.759457 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-fd1a0575-3ef4-4e3c-80d0-b85576e1206d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:10.760051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-fd1a0575-3ef4-4e3c-80d0-b85576e1206d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:10.766356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fd1a0575-3ef4-4e3c-80d0-b85576e1206d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:10.766801 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fd1a0575-3ef4-4e3c-80d0-b85576e1206d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:10.767555 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fd1a0575-3ef4-4e3c-80d0-b85576e1206d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:10.767984 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fd1a0575-3ef4-4e3c-80d0-b85576e1206d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:10.894972 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-fd1a0575-3ef4-4e3c-80d0-b85576e1206d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:10.902906 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-fd1a0575-3ef4-4e3c-80d0-b85576e1206d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146955 Jun 10 13:17:10.903468 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 749/1376] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:10 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:11.682057 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5b3a64bc-4c8e-4f52-878e-6aef5387c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:11.682663 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5b3a64bc-4c8e-4f52-878e-6aef5387c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:11.689810 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5b3a64bc-4c8e-4f52-878e-6aef5387c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:11.690253 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5b3a64bc-4c8e-4f52-878e-6aef5387c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:11.691001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5b3a64bc-4c8e-4f52-878e-6aef5387c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:11.691432 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5b3a64bc-4c8e-4f52-878e-6aef5387c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:11.820356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5b3a64bc-4c8e-4f52-878e-6aef5387c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:11.828030 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5b3a64bc-4c8e-4f52-878e-6aef5387c313 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149924 Jun 10 13:17:11.828590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 750/1377] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:11 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:11.920578 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8ad13647-9be4-45ac-9562-44e3d23f9bdf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:11.921758 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8ad13647-9be4-45ac-9562-44e3d23f9bdf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:11.927707 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8ad13647-9be4-45ac-9562-44e3d23f9bdf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:11.928150 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8ad13647-9be4-45ac-9562-44e3d23f9bdf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:11.928929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8ad13647-9be4-45ac-9562-44e3d23f9bdf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:11.929358 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8ad13647-9be4-45ac-9562-44e3d23f9bdf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:12.052699 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8ad13647-9be4-45ac-9562-44e3d23f9bdf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:12.060517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8ad13647-9be4-45ac-9562-44e3d23f9bdf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144387 Jun 10 13:17:12.061175 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 751/1378] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:11 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:12.845583 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2e7f7390-3f4d-49f0-b77b-24da4ee0ec48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:12.846257 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2e7f7390-3f4d-49f0-b77b-24da4ee0ec48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:12.852675 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2e7f7390-3f4d-49f0-b77b-24da4ee0ec48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:12.853152 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2e7f7390-3f4d-49f0-b77b-24da4ee0ec48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:12.853903 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2e7f7390-3f4d-49f0-b77b-24da4ee0ec48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:12.854338 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2e7f7390-3f4d-49f0-b77b-24da4ee0ec48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:12.979453 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2e7f7390-3f4d-49f0-b77b-24da4ee0ec48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:12.987688 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2e7f7390-3f4d-49f0-b77b-24da4ee0ec48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146462 Jun 10 13:17:12.988255 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 752/1379] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:12 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:13.077981 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-96774bb1-d12e-432a-b82c-6ce37fdf99df tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:13.078588 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-96774bb1-d12e-432a-b82c-6ce37fdf99df tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:13.084454 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-96774bb1-d12e-432a-b82c-6ce37fdf99df tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:13.085033 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-96774bb1-d12e-432a-b82c-6ce37fdf99df tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:13.085793 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-96774bb1-d12e-432a-b82c-6ce37fdf99df tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:13.086225 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-96774bb1-d12e-432a-b82c-6ce37fdf99df tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:13.215222 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-96774bb1-d12e-432a-b82c-6ce37fdf99df tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:13.223212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-96774bb1-d12e-432a-b82c-6ce37fdf99df tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.149639 Jun 10 13:17:13.223776 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 753/1380] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:13 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:14.003958 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8410f698-3788-48b4-a058-e0632d6b5265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:14.004638 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8410f698-3788-48b4-a058-e0632d6b5265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:14.010955 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8410f698-3788-48b4-a058-e0632d6b5265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:14.011399 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8410f698-3788-48b4-a058-e0632d6b5265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:14.012147 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8410f698-3788-48b4-a058-e0632d6b5265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:14.012572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8410f698-3788-48b4-a058-e0632d6b5265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:14.142569 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8410f698-3788-48b4-a058-e0632d6b5265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:14.150744 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8410f698-3788-48b4-a058-e0632d6b5265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.150654 Jun 10 13:17:14.151310 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 754/1381] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:13 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:14.303448 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1197ffa2-9572-426b-885c-aeb733dd6991 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:14.304056 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1197ffa2-9572-426b-885c-aeb733dd6991 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:14.309975 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1197ffa2-9572-426b-885c-aeb733dd6991 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:14.310946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1197ffa2-9572-426b-885c-aeb733dd6991 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:14.311687 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1197ffa2-9572-426b-885c-aeb733dd6991 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:14.312110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1197ffa2-9572-426b-885c-aeb733dd6991 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:14.441205 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1197ffa2-9572-426b-885c-aeb733dd6991 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:14.450124 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1197ffa2-9572-426b-885c-aeb733dd6991 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.213177 Jun 10 13:17:14.451773 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 755/1382] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:14 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:15.169090 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d6d10ca4-d47d-4274-9be2-e36fe5c01bab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:15.170329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d6d10ca4-d47d-4274-9be2-e36fe5c01bab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:15.176784 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d6d10ca4-d47d-4274-9be2-e36fe5c01bab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:15.177240 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d6d10ca4-d47d-4274-9be2-e36fe5c01bab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:15.177991 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d6d10ca4-d47d-4274-9be2-e36fe5c01bab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:15.178422 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d6d10ca4-d47d-4274-9be2-e36fe5c01bab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:15.181040 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:INSERT=1,nova_cell1:SELECT=99 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:17:15.307979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d6d10ca4-d47d-4274-9be2-e36fe5c01bab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:17:15.316006 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d6d10ca4-d47d-4274-9be2-e36fe5c01bab tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.150882 Jun 10 13:17:15.316568 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 756/1383] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:17:15 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:17:15.386528 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-94e71739-efa7-4c11-89c7-c05e6c025c1f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:17:15.387064 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-94e71739-efa7-4c11-89c7-c05e6c025c1f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:17:15.393006 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-94e71739-efa7-4c11-89c7-c05e6c025c1f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:15.393451 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-94e71739-efa7-4c11-89c7-c05e6c025c1f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:15.394312 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-94e71739-efa7-4c11-89c7-c05e6c025c1f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:17:15.394774 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-94e71739-efa7-4c11-89c7-c05e6c025c1f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:17:17.934187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-e13a9757-eaec-40f1-b7c4-16d8eb4a8147 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lazy-loading 'services' on Instance uuid a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:17:18.088262 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-94e71739-efa7-4c11-89c7-c05e6c025c1f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lazy-loading 'services' on Instance uuid a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:17:25.390620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=52,nova_api:UPDATE=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:17:27.971159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=4 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:17:28.092819 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=13 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:18:15.489586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e13a9757-eaec-40f1-b7c4-16d8eb4a8147 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e" status: 202 len: 0 microversion: 2.1 time: 120.380089 Jun 10 13:18:15.490581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 628/1384] 10.222.0.25 () {58 vars in 1415 bytes} [Fri Jun 10 13:16:15 2022] DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e => generated 0 bytes in 120382 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:18:15.496501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-032c83c4-2bda-4c8f-a059-c0219321bf05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:15.497218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-032c83c4-2bda-4c8f-a059-c0219321bf05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:15.503531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-032c83c4-2bda-4c8f-a059-c0219321bf05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:15.503972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-032c83c4-2bda-4c8f-a059-c0219321bf05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:15.504809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-032c83c4-2bda-4c8f-a059-c0219321bf05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:15.505191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-032c83c4-2bda-4c8f-a059-c0219321bf05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:15.561950 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.wsgi [None req-94e71739-efa7-4c11-89c7-c05e6c025c1f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c5aea33-ac3d-4a7a-8315-20ce5250de06) Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-94e71739-efa7-4c11-89c7-c05e6c025c1f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Request method failure captured: Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: request: DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e HTTP/1.1 Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept: application/json Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept-Encoding: identity Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Connection: close Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Length: 0 Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Type: application/json Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Host: 10.222.0.25 Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Openstack-System-Scope: None Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: User-Agent: python-urllib3/1.26.9 Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Auth-Token: gAAAAABio0HlyICQX9l5KFYyUsFzMtUgTAzPRfxQzNN7dM9kV13g0iWQTn9QMENYtfZqE20ETOnCUUxvj1yudvSxd1bFTvMSIvc_eVAfoGvb2Phk1I--eUGA8-ePhD-Rp01rKEhzSDpYm-ppUB1HJXel5mUDpZwxED5NTdU7SM0nV7mlEGF1wbo Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Id: None Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Name: None Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Identity-Status: Confirmed Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Is-Admin-Project: True Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Id: default Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Name: Default Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Id: ff2616337ae440a89047fd9a50218387 Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Role: reader,member Jun 10 13:18:15.564279 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Roles: reader,member Jun 10 13:18:15.565709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ff2616337ae440a89047fd9a50218387"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ff2616337ae440a89047fd9a50218387"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ff2616337ae440a89047fd9a50218387", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ff2616337ae440a89047fd9a50218387"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:18:15.565709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:18:15.565709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Id: ff2616337ae440a89047fd9a50218387 Jun 10 13:18:15.565709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:18:15.565709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User: tempest-VolumesSnapshotTestJSON-1698707508-project Jun 10 13:18:15.565709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Id: default Jun 10 13:18:15.565709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Name: Default Jun 10 13:18:15.565709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Id: 849e36f8653d4d1f9a7fb1df8fb913a9 Jun 10 13:18:15.565709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Name: tempest-VolumesSnapshotTestJSON-1698707508-project Jun 10 13:18:15.565709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: method: > Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c5aea33-ac3d-4a7a-8315-20ce5250de06) Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: action_args: {'server_id': 'a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744', 'id': '802588c0-10d5-4ca0-8cbb-585dee61c94e'} Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jun 10 13:18:15.566608 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c5aea33-ac3d-4a7a-8315-20ce5250de06) Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:18:15.567772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c5aea33-ac3d-4a7a-8315-20ce5250de06) Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:18:15.568979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c5aea33-ac3d-4a7a-8315-20ce5250de06) Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:18:15.570001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c5aea33-ac3d-4a7a-8315-20ce5250de06) Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c5aea33-ac3d-4a7a-8315-20ce5250de06) Jun 10 13:18:15.571198 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:18:15.572262 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-94e71739-efa7-4c11-89c7-c05e6c025c1f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c5aea33-ac3d-4a7a-8315-20ce5250de06) {{(pid=118093) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:18:15.572262 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-94e71739-efa7-4c11-89c7-c05e6c025c1f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e" status: 400 len: 269 microversion: 2.1 time: 60.247506 Jun 10 13:18:15.572262 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 757/1385] 10.222.0.25 () {58 vars in 1415 bytes} [Fri Jun 10 13:17:15 2022] DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e => generated 269 bytes in 60249 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jun 10 13:18:15.670164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-032c83c4-2bda-4c8f-a059-c0219321bf05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:15.682205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-032c83c4-2bda-4c8f-a059-c0219321bf05 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.188453 Jun 10 13:18:15.682205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 629/1386] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:15 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:15.685927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3c276f50-871f-42ac-8f93-e434696a4eac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:15.686484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3c276f50-871f-42ac-8f93-e434696a4eac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:15.692993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c276f50-871f-42ac-8f93-e434696a4eac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:15.693506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c276f50-871f-42ac-8f93-e434696a4eac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:15.694409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c276f50-871f-42ac-8f93-e434696a4eac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:15.694905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c276f50-871f-42ac-8f93-e434696a4eac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:15.719865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3c276f50-871f-42ac-8f93-e434696a4eac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:18:15.722673 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b11a8502-6ffc-46eb-9b7f-9976fccc33f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:15.723339 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b11a8502-6ffc-46eb-9b7f-9976fccc33f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:15.729371 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b11a8502-6ffc-46eb-9b7f-9976fccc33f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:15.729818 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b11a8502-6ffc-46eb-9b7f-9976fccc33f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:15.730581 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b11a8502-6ffc-46eb-9b7f-9976fccc33f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:15.731013 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b11a8502-6ffc-46eb-9b7f-9976fccc33f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:15.882424 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b11a8502-6ffc-46eb-9b7f-9976fccc33f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:15.890785 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b11a8502-6ffc-46eb-9b7f-9976fccc33f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.322560 Jun 10 13:18:15.892041 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 758/1387] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:15 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:15.895995 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6773613a-6114-437b-935e-0e3e8bece4be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:15.896475 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6773613a-6114-437b-935e-0e3e8bece4be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:15.902127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6773613a-6114-437b-935e-0e3e8bece4be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:15.902566 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6773613a-6114-437b-935e-0e3e8bece4be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:15.903310 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6773613a-6114-437b-935e-0e3e8bece4be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:15.903734 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6773613a-6114-437b-935e-0e3e8bece4be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:16.072317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3c276f50-871f-42ac-8f93-e434696a4eac tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "DELETE /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 204 len: 0 microversion: 2.1 time: 0.390343 Jun 10 13:18:16.072936 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 630/1388] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:18:15 2022] DELETE /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jun 10 13:18:16.078348 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ff56bb88-d09b-4abd-9f76-54b9d9b45dba tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:16.079017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ff56bb88-d09b-4abd-9f76-54b9d9b45dba tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:16.084727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff56bb88-d09b-4abd-9f76-54b9d9b45dba tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:16.086941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff56bb88-d09b-4abd-9f76-54b9d9b45dba tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:16.086941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff56bb88-d09b-4abd-9f76-54b9d9b45dba tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:16.086941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff56bb88-d09b-4abd-9f76-54b9d9b45dba tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:16.214153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ff56bb88-d09b-4abd-9f76-54b9d9b45dba tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:16.222107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ff56bb88-d09b-4abd-9f76-54b9d9b45dba tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147993 Jun 10 13:18:16.222673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 631/1389] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:16 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:16.227135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4d92d466-4870-47c6-86ac-023cabf19220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:16.227737 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4d92d466-4870-47c6-86ac-023cabf19220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:16.234305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4d92d466-4870-47c6-86ac-023cabf19220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:16.234753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4d92d466-4870-47c6-86ac-023cabf19220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:16.235509 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4d92d466-4870-47c6-86ac-023cabf19220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:16.235944 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4d92d466-4870-47c6-86ac-023cabf19220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:16.364059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4d92d466-4870-47c6-86ac-023cabf19220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:16.371419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4d92d466-4870-47c6-86ac-023cabf19220 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1674 microversion: 2.1 time: 0.147822 Jun 10 13:18:16.372651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 632/1390] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:16 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1674 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:16.908122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a42fb30d-27a7-4604-9bdb-f45c4cfa8420 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:16.908783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a42fb30d-27a7-4604-9bdb-f45c4cfa8420 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:16.914823 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a42fb30d-27a7-4604-9bdb-f45c4cfa8420 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:16.915260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a42fb30d-27a7-4604-9bdb-f45c4cfa8420 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:16.916011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a42fb30d-27a7-4604-9bdb-f45c4cfa8420 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:16.916434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a42fb30d-27a7-4604-9bdb-f45c4cfa8420 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:17.044079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a42fb30d-27a7-4604-9bdb-f45c4cfa8420 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:17.051691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a42fb30d-27a7-4604-9bdb-f45c4cfa8420 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.148170 Jun 10 13:18:17.052253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 633/1391] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:16 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:17.239498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a16e4925-6abe-43e8-aba2-275aa006c03c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:17.240060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a16e4925-6abe-43e8-aba2-275aa006c03c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:17.246300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a16e4925-6abe-43e8-aba2-275aa006c03c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:17.246749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a16e4925-6abe-43e8-aba2-275aa006c03c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:17.248027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a16e4925-6abe-43e8-aba2-275aa006c03c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:17.248470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a16e4925-6abe-43e8-aba2-275aa006c03c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:17.384429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a16e4925-6abe-43e8-aba2-275aa006c03c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:17.393000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a16e4925-6abe-43e8-aba2-275aa006c03c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.157433 Jun 10 13:18:17.393574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 634/1392] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:17 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:17.397945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-66b28a0e-d3b0-4e2a-877b-8f95719e5ed1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:17.398543 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-66b28a0e-d3b0-4e2a-877b-8f95719e5ed1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:17.404574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-66b28a0e-d3b0-4e2a-877b-8f95719e5ed1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:17.404746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-66b28a0e-d3b0-4e2a-877b-8f95719e5ed1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:17.405548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-66b28a0e-d3b0-4e2a-877b-8f95719e5ed1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:17.405982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-66b28a0e-d3b0-4e2a-877b-8f95719e5ed1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:17.538464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-66b28a0e-d3b0-4e2a-877b-8f95719e5ed1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:17.546536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-66b28a0e-d3b0-4e2a-877b-8f95719e5ed1 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1674 microversion: 2.1 time: 0.151991 Jun 10 13:18:17.551891 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 635/1393] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:17 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1674 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:17.774050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-98aaefcb-ec4a-4472-b7a5-104616dc0fea req-89292265-e479-418e-9e1d-fa150fb2ecfa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6273d596-6d75-49f6-9c03-17ea7b4422d7", "name": "network-vif-deleted", "tag": "d939095b-bad6-443e-ac1d-07e8dd47e777"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:18:17.782345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-98aaefcb-ec4a-4472-b7a5-104616dc0fea req-89292265-e479-418e-9e1d-fa150fb2ecfa service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:17.782788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-98aaefcb-ec4a-4472-b7a5-104616dc0fea req-89292265-e479-418e-9e1d-fa150fb2ecfa service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:17.794869 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-98aaefcb-ec4a-4472-b7a5-104616dc0fea req-89292265-e479-418e-9e1d-fa150fb2ecfa service nova] Creating event network-vif-deleted:d939095b-bad6-443e-ac1d-07e8dd47e777 for instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 on hv-cinder-maste Jun 10 13:18:17.800298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-98aaefcb-ec4a-4472-b7a5-104616dc0fea req-89292265-e479-418e-9e1d-fa150fb2ecfa service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029924 Jun 10 13:18:17.800910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 636/1394] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:18:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:18:18.069624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a728bc84-6da9-4013-81eb-5b42148e0cdc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:18.072077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a728bc84-6da9-4013-81eb-5b42148e0cdc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:18.077099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a728bc84-6da9-4013-81eb-5b42148e0cdc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:18.077546 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a728bc84-6da9-4013-81eb-5b42148e0cdc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:18.078544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a728bc84-6da9-4013-81eb-5b42148e0cdc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:18.078975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a728bc84-6da9-4013-81eb-5b42148e0cdc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:18.236787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a728bc84-6da9-4013-81eb-5b42148e0cdc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:18.244686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a728bc84-6da9-4013-81eb-5b42148e0cdc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.179249 Jun 10 13:18:18.245456 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 637/1395] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:18 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:18.410270 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5187964c-6363-461c-a669-ea5bbdb0e920 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:18.410870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5187964c-6363-461c-a669-ea5bbdb0e920 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:18.416750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5187964c-6363-461c-a669-ea5bbdb0e920 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:18.417218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5187964c-6363-461c-a669-ea5bbdb0e920 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:18.417962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5187964c-6363-461c-a669-ea5bbdb0e920 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:18.418387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5187964c-6363-461c-a669-ea5bbdb0e920 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:18.739200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5187964c-6363-461c-a669-ea5bbdb0e920 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:18.749358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5187964c-6363-461c-a669-ea5bbdb0e920 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.342818 Jun 10 13:18:18.749941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 638/1396] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:18 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:18.754705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5a3af744-a756-4de4-8ac5-6acfd311780f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:18.755308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5a3af744-a756-4de4-8ac5-6acfd311780f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:18.761194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5a3af744-a756-4de4-8ac5-6acfd311780f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:18.761662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5a3af744-a756-4de4-8ac5-6acfd311780f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:18.763040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5a3af744-a756-4de4-8ac5-6acfd311780f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:18.763487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5a3af744-a756-4de4-8ac5-6acfd311780f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:18.793568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5a3af744-a756-4de4-8ac5-6acfd311780f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:18.794074 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5a3af744-a756-4de4-8ac5-6acfd311780f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:18.794560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-5a3af744-a756-4de4-8ac5-6acfd311780f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lazy-loading 'fault' on Instance uuid 6273d596-6d75-49f6-9c03-17ea7b4422d7 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:18:18.834968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5a3af744-a756-4de4-8ac5-6acfd311780f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c8b2c27b69a4776ab6b7b7820da02d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6dace38cecd44f87becbf6dc213851b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:18.847799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5a3af744-a756-4de4-8ac5-6acfd311780f tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 200 len: 1327 microversion: 2.1 time: 0.092708 Jun 10 13:18:18.847799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 639/1397] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:18 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 1327 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:19.261428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c33478cf-6f4b-43bb-aa4b-7a03c80d68e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:19.262081 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c33478cf-6f4b-43bb-aa4b-7a03c80d68e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:19.268456 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c33478cf-6f4b-43bb-aa4b-7a03c80d68e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:19.268955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c33478cf-6f4b-43bb-aa4b-7a03c80d68e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:19.269804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c33478cf-6f4b-43bb-aa4b-7a03c80d68e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:19.270145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c33478cf-6f4b-43bb-aa4b-7a03c80d68e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:19.400374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c33478cf-6f4b-43bb-aa4b-7a03c80d68e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:19.408454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c33478cf-6f4b-43bb-aa4b-7a03c80d68e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.151006 Jun 10 13:18:19.409072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 640/1398] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:19 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:19.767036 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d7b446e6-7da0-4976-b5fd-41f7551eb499 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:19.767641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d7b446e6-7da0-4976-b5fd-41f7551eb499 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:19.773699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d7b446e6-7da0-4976-b5fd-41f7551eb499 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:19.774141 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d7b446e6-7da0-4976-b5fd-41f7551eb499 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:19.774896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d7b446e6-7da0-4976-b5fd-41f7551eb499 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:19.775323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d7b446e6-7da0-4976-b5fd-41f7551eb499 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:19.909675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d7b446e6-7da0-4976-b5fd-41f7551eb499 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:19.917477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d7b446e6-7da0-4976-b5fd-41f7551eb499 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.154833 Jun 10 13:18:19.918037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 641/1399] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:19 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:19.922312 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cab3ed34-4299-40f9-8039-933a90da5300 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:19.922888 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cab3ed34-4299-40f9-8039-933a90da5300 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:19.928322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cab3ed34-4299-40f9-8039-933a90da5300 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:19.928785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cab3ed34-4299-40f9-8039-933a90da5300 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:19.929535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cab3ed34-4299-40f9-8039-933a90da5300 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:19.929959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cab3ed34-4299-40f9-8039-933a90da5300 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:19.942839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-cab3ed34-4299-40f9-8039-933a90da5300 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] HTTP exception thrown: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cab3ed34-4299-40f9-8039-933a90da5300 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Request method failure captured: Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 HTTP/1.1 Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0GLIJPQ8lSWSCxXU9neIcS9fWNdW4XJF-omQVn8xvgfajtrPtR9xT7yJRQ7uDkAeVSjjX8Zr-TojYB67_YvghXuxNAaWo9VFXLD41gtOICuurNBjdxNv9Y58tQmvsCSdX2FomGdBsssQdaFXDYnlp0d7vxIpJLVub7cC52vynSBvwE Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 6dace38cecd44f87becbf6dc213851b3 Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:18:19.944425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:18:19.946043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_6dace38cecd44f87becbf6dc213851b3", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:18:19.946043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:18:19.946043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 6dace38cecd44f87becbf6dc213851b3 Jun 10 13:18:19.946043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:18:19.946043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeTestJSON-1599090335-project Jun 10 13:18:19.946043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:18:19.946043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:18:19.946043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 9c8b2c27b69a4776ab6b7b7820da02d9 Jun 10 13:18:19.946043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeTestJSON-1599090335-project Jun 10 13:18:19.946043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:18:19.946043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:18:19.946043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '6273d596-6d75-49f6-9c03-17ea7b4422d7'} Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:18:19.946958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:18:19.948570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:18:19.950549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cab3ed34-4299-40f9-8039-933a90da5300 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Returning 404 to user: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:18:19.950549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cab3ed34-4299-40f9-8039-933a90da5300 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 404 len: 111 microversion: 2.1 time: 0.026855 Jun 10 13:18:19.950549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 642/1400] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:19 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 111 bytes in 28 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:18:19.955281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5a5bb106-52c6-4590-a6b7-ca435cee007e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '.version_select at 0x7f930544d700>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:19.958431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR oslo.messaging._drivers.impl_rabbit [None req-5a5bb106-52c6-4590-a6b7-ca435cee007e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [0cbbd196-2faa-47ef-826b-fce2a888b47f] AMQP server on 10.222.0.25:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jun 10 13:18:20.966526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [None req-5a5bb106-52c6-4590-a6b7-ca435cee007e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [0cbbd196-2faa-47ef-826b-fce2a888b47f] Reconnected to AMQP server on 10.222.0.25:5672 via [amqp] client with port 38080. Jun 10 13:18:21.045145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5a5bb106-52c6-4590-a6b7-ca435cee007e tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-513401585" status: 202 len: 0 microversion: 2.1 time: 1.093162 Jun 10 13:18:21.045747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 643/1401] 10.222.0.25 () {58 vars in 1217 bytes} [Fri Jun 10 13:18:19 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-513401585 => generated 0 bytes in 1094 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:18:21.050266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0a62416e-2390-4f7e-9f3f-0fc30073ae87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:21.050894 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0a62416e-2390-4f7e-9f3f-0fc30073ae87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:21.057543 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0a62416e-2390-4f7e-9f3f-0fc30073ae87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:21.057996 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0a62416e-2390-4f7e-9f3f-0fc30073ae87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:21.058761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0a62416e-2390-4f7e-9f3f-0fc30073ae87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:21.059195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0a62416e-2390-4f7e-9f3f-0fc30073ae87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:21.189088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0a62416e-2390-4f7e-9f3f-0fc30073ae87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:21.196549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0a62416e-2390-4f7e-9f3f-0fc30073ae87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149867 Jun 10 13:18:21.197151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 644/1402] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:21 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:21.205024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-494f0b78-652c-40bf-b3e5-fff9d82d9377 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:21.205751 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-494f0b78-652c-40bf-b3e5-fff9d82d9377 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:21.214985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-494f0b78-652c-40bf-b3e5-fff9d82d9377 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:21.216089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-494f0b78-652c-40bf-b3e5-fff9d82d9377 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:21.216988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-494f0b78-652c-40bf-b3e5-fff9d82d9377 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:21.217358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-494f0b78-652c-40bf-b3e5-fff9d82d9377 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:21.235964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:18:21.358352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-494f0b78-652c-40bf-b3e5-fff9d82d9377 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:21.366672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-494f0b78-652c-40bf-b3e5-fff9d82d9377 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.168489 Jun 10 13:18:21.367332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 645/1403] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:21 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:22.214206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-787c5020-9c35-45b0-874b-bd92ebdf8596 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:22.214799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-787c5020-9c35-45b0-874b-bd92ebdf8596 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:22.221379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-787c5020-9c35-45b0-874b-bd92ebdf8596 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:22.221820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-787c5020-9c35-45b0-874b-bd92ebdf8596 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:22.222570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-787c5020-9c35-45b0-874b-bd92ebdf8596 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:22.222996 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-787c5020-9c35-45b0-874b-bd92ebdf8596 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:22.351028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-787c5020-9c35-45b0-874b-bd92ebdf8596 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:22.359133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-787c5020-9c35-45b0-874b-bd92ebdf8596 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.148754 Jun 10 13:18:22.359702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 646/1404] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:22 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:22.382713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9c7cac5e-037e-4280-b867-611a80ca0a71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:22.383308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9c7cac5e-037e-4280-b867-611a80ca0a71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:22.388961 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c7cac5e-037e-4280-b867-611a80ca0a71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:22.389408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c7cac5e-037e-4280-b867-611a80ca0a71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:22.390152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c7cac5e-037e-4280-b867-611a80ca0a71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:22.390578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c7cac5e-037e-4280-b867-611a80ca0a71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:22.515167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9c7cac5e-037e-4280-b867-611a80ca0a71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:22.522896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9c7cac5e-037e-4280-b867-611a80ca0a71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143652 Jun 10 13:18:22.523455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 647/1405] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:22 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:23.376571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c98f42ec-8a8f-4198-98a8-6ab41e84b9d0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:23.377207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c98f42ec-8a8f-4198-98a8-6ab41e84b9d0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:23.383636 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c98f42ec-8a8f-4198-98a8-6ab41e84b9d0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:23.384113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c98f42ec-8a8f-4198-98a8-6ab41e84b9d0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:23.384885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c98f42ec-8a8f-4198-98a8-6ab41e84b9d0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:23.385323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c98f42ec-8a8f-4198-98a8-6ab41e84b9d0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:23.510210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c98f42ec-8a8f-4198-98a8-6ab41e84b9d0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:23.518139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c98f42ec-8a8f-4198-98a8-6ab41e84b9d0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145440 Jun 10 13:18:23.518701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 648/1406] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:23 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:23.538643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6897bcfe-1952-4d45-b5d7-9ec723124c95 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:23.539263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6897bcfe-1952-4d45-b5d7-9ec723124c95 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:23.545320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6897bcfe-1952-4d45-b5d7-9ec723124c95 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:23.545760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6897bcfe-1952-4d45-b5d7-9ec723124c95 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:23.546506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6897bcfe-1952-4d45-b5d7-9ec723124c95 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:23.546932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6897bcfe-1952-4d45-b5d7-9ec723124c95 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:23.672323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6897bcfe-1952-4d45-b5d7-9ec723124c95 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:23.680223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6897bcfe-1952-4d45-b5d7-9ec723124c95 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144982 Jun 10 13:18:23.680809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 649/1407] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:23 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:24.534983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4fa75f54-60e7-4f60-aecc-5dfce81860de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:24.535625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4fa75f54-60e7-4f60-aecc-5dfce81860de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:24.542140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4fa75f54-60e7-4f60-aecc-5dfce81860de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:24.542583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4fa75f54-60e7-4f60-aecc-5dfce81860de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:24.543328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4fa75f54-60e7-4f60-aecc-5dfce81860de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:24.543756 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4fa75f54-60e7-4f60-aecc-5dfce81860de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:24.668380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4fa75f54-60e7-4f60-aecc-5dfce81860de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:24.675804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4fa75f54-60e7-4f60-aecc-5dfce81860de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.144641 Jun 10 13:18:24.676362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 650/1408] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:24 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:24.697230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-12f0310f-23b4-47ce-91c8-1b69d6b2e3cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:24.697824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-12f0310f-23b4-47ce-91c8-1b69d6b2e3cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:24.703907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-12f0310f-23b4-47ce-91c8-1b69d6b2e3cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:24.704353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-12f0310f-23b4-47ce-91c8-1b69d6b2e3cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:24.705129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-12f0310f-23b4-47ce-91c8-1b69d6b2e3cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:24.705563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-12f0310f-23b4-47ce-91c8-1b69d6b2e3cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:24.828852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-12f0310f-23b4-47ce-91c8-1b69d6b2e3cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:24.836611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-12f0310f-23b4-47ce-91c8-1b69d6b2e3cd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.142821 Jun 10 13:18:24.837416 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 651/1409] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:24 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:25.311667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:18:25.312251 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Fri Jun 10 13:18:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e (ip 10.222.0.25) !!! Jun 10 13:18:25.692556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-067dacc4-c6c0-4c5b-9ab8-2e64abccf276 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:25.693231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-067dacc4-c6c0-4c5b-9ab8-2e64abccf276 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:25.700050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-067dacc4-c6c0-4c5b-9ab8-2e64abccf276 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:25.700489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-067dacc4-c6c0-4c5b-9ab8-2e64abccf276 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:25.701269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-067dacc4-c6c0-4c5b-9ab8-2e64abccf276 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:25.701696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-067dacc4-c6c0-4c5b-9ab8-2e64abccf276 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:25.833093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-067dacc4-c6c0-4c5b-9ab8-2e64abccf276 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:25.840570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-067dacc4-c6c0-4c5b-9ab8-2e64abccf276 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.151810 Jun 10 13:18:25.841161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 652/1410] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:25 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:25.852800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8ce4959f-fd59-4d47-9bcd-947ad35ef8c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:25.853410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8ce4959f-fd59-4d47-9bcd-947ad35ef8c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:25.859313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ce4959f-fd59-4d47-9bcd-947ad35ef8c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:25.859753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ce4959f-fd59-4d47-9bcd-947ad35ef8c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:25.860495 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ce4959f-fd59-4d47-9bcd-947ad35ef8c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:25.860950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ce4959f-fd59-4d47-9bcd-947ad35ef8c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:25.900129 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=4 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:18:25.921484 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=10 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:18:25.983889 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8ce4959f-fd59-4d47-9bcd-947ad35ef8c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:25.991698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8ce4959f-fd59-4d47-9bcd-947ad35ef8c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.142374 Jun 10 13:18:25.992457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 653/1411] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:25 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:26.857950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-82f6c81b-62f4-463a-a462-248bdf54ed5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:26.859161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-82f6c81b-62f4-463a-a462-248bdf54ed5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:26.865629 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-82f6c81b-62f4-463a-a462-248bdf54ed5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:26.866112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-82f6c81b-62f4-463a-a462-248bdf54ed5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:26.866895 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-82f6c81b-62f4-463a-a462-248bdf54ed5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:26.867313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-82f6c81b-62f4-463a-a462-248bdf54ed5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:26.996134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-82f6c81b-62f4-463a-a462-248bdf54ed5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:27.003580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-82f6c81b-62f4-463a-a462-248bdf54ed5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149765 Jun 10 13:18:27.004233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 654/1412] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:26 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:27.008720 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-19ac477f-0a3c-4b01-bd3d-957908e53c2e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:27.009364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-19ac477f-0a3c-4b01-bd3d-957908e53c2e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:27.015208 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-19ac477f-0a3c-4b01-bd3d-957908e53c2e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:27.015658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-19ac477f-0a3c-4b01-bd3d-957908e53c2e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:27.016425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-19ac477f-0a3c-4b01-bd3d-957908e53c2e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:27.016905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-19ac477f-0a3c-4b01-bd3d-957908e53c2e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:27.141124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-19ac477f-0a3c-4b01-bd3d-957908e53c2e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:27.149159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-19ac477f-0a3c-4b01-bd3d-957908e53c2e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143807 Jun 10 13:18:27.149727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 655/1413] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:27 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:28.020477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cabac618-5b6f-4765-b194-a3718ec258ee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:28.021174 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cabac618-5b6f-4765-b194-a3718ec258ee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:28.027623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cabac618-5b6f-4765-b194-a3718ec258ee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:28.028066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cabac618-5b6f-4765-b194-a3718ec258ee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:28.028851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cabac618-5b6f-4765-b194-a3718ec258ee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:28.029286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cabac618-5b6f-4765-b194-a3718ec258ee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:28.164082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cabac618-5b6f-4765-b194-a3718ec258ee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:28.168218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cabac618-5b6f-4765-b194-a3718ec258ee tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.151544 Jun 10 13:18:28.168810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 656/1414] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:28 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:28.173632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3d7076c3-a2a2-498c-980f-baa69174f47a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:28.174222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3d7076c3-a2a2-498c-980f-baa69174f47a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:28.179747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3d7076c3-a2a2-498c-980f-baa69174f47a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:28.180192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3d7076c3-a2a2-498c-980f-baa69174f47a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:28.180968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3d7076c3-a2a2-498c-980f-baa69174f47a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:28.181397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3d7076c3-a2a2-498c-980f-baa69174f47a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:28.309446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3d7076c3-a2a2-498c-980f-baa69174f47a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:28.318039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3d7076c3-a2a2-498c-980f-baa69174f47a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148289 Jun 10 13:18:28.318603 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 657/1415] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:28 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:29.185624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e7f77c02-fe28-45e0-9179-17b580ec5ed8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:29.186273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e7f77c02-fe28-45e0-9179-17b580ec5ed8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:29.193540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7f77c02-fe28-45e0-9179-17b580ec5ed8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:29.193988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7f77c02-fe28-45e0-9179-17b580ec5ed8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:29.194754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7f77c02-fe28-45e0-9179-17b580ec5ed8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:29.195197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7f77c02-fe28-45e0-9179-17b580ec5ed8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:29.320370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e7f77c02-fe28-45e0-9179-17b580ec5ed8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:29.328003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e7f77c02-fe28-45e0-9179-17b580ec5ed8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146175 Jun 10 13:18:29.328611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 658/1416] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:29 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:29.334139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0fc34211-d3e6-4a6f-ba4e-bdec8a835ee9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:29.334845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0fc34211-d3e6-4a6f-ba4e-bdec8a835ee9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:29.340760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fc34211-d3e6-4a6f-ba4e-bdec8a835ee9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:29.341308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fc34211-d3e6-4a6f-ba4e-bdec8a835ee9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:29.342061 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fc34211-d3e6-4a6f-ba4e-bdec8a835ee9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:29.342491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fc34211-d3e6-4a6f-ba4e-bdec8a835ee9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:29.469784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0fc34211-d3e6-4a6f-ba4e-bdec8a835ee9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:29.477668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0fc34211-d3e6-4a6f-ba4e-bdec8a835ee9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147013 Jun 10 13:18:29.478237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 659/1417] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:29 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:30.344903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9ee4c57d-c65e-402e-81c7-eafe5ef3e7ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:30.345507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9ee4c57d-c65e-402e-81c7-eafe5ef3e7ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:30.351879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ee4c57d-c65e-402e-81c7-eafe5ef3e7ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:30.352317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ee4c57d-c65e-402e-81c7-eafe5ef3e7ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:30.353087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ee4c57d-c65e-402e-81c7-eafe5ef3e7ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:30.353509 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ee4c57d-c65e-402e-81c7-eafe5ef3e7ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:30.483208 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9ee4c57d-c65e-402e-81c7-eafe5ef3e7ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:30.490589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9ee4c57d-c65e-402e-81c7-eafe5ef3e7ff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149526 Jun 10 13:18:30.491149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 660/1418] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:30 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:30.495520 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c9f8c4d4-d0ec-4876-a747-506739fd25f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:30.496102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c9f8c4d4-d0ec-4876-a747-506739fd25f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:30.502118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9f8c4d4-d0ec-4876-a747-506739fd25f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:30.502567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9f8c4d4-d0ec-4876-a747-506739fd25f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:30.503323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9f8c4d4-d0ec-4876-a747-506739fd25f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:30.503755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9f8c4d4-d0ec-4876-a747-506739fd25f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:30.634269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c9f8c4d4-d0ec-4876-a747-506739fd25f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:30.642094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c9f8c4d4-d0ec-4876-a747-506739fd25f0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.150016 Jun 10 13:18:30.642651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 661/1419] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:30 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:31.507652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-435593be-64a4-46a3-935d-153e4c0ab07d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:31.508318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-435593be-64a4-46a3-935d-153e4c0ab07d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:31.514400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-435593be-64a4-46a3-935d-153e4c0ab07d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:31.514841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-435593be-64a4-46a3-935d-153e4c0ab07d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:31.515603 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-435593be-64a4-46a3-935d-153e4c0ab07d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:31.516029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-435593be-64a4-46a3-935d-153e4c0ab07d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:31.519957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:18:31.642146 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-435593be-64a4-46a3-935d-153e4c0ab07d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:31.649679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-435593be-64a4-46a3-935d-153e4c0ab07d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145859 Jun 10 13:18:31.650241 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 662/1420] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:31 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:31.658448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-04f9eb83-2ef6-4965-9453-e6390d88e837 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:31.659044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-04f9eb83-2ef6-4965-9453-e6390d88e837 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:31.664732 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04f9eb83-2ef6-4965-9453-e6390d88e837 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:31.665205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04f9eb83-2ef6-4965-9453-e6390d88e837 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:31.665950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04f9eb83-2ef6-4965-9453-e6390d88e837 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:31.666375 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04f9eb83-2ef6-4965-9453-e6390d88e837 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:31.816120 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-04f9eb83-2ef6-4965-9453-e6390d88e837 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:31.824323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-04f9eb83-2ef6-4965-9453-e6390d88e837 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.169386 Jun 10 13:18:31.824965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 663/1421] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:31 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:32.666735 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-44218f7c-aca8-443c-aa81-38189b3437ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:32.667401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-44218f7c-aca8-443c-aa81-38189b3437ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:32.673796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-44218f7c-aca8-443c-aa81-38189b3437ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:32.674234 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-44218f7c-aca8-443c-aa81-38189b3437ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:32.674977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-44218f7c-aca8-443c-aa81-38189b3437ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:32.675399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-44218f7c-aca8-443c-aa81-38189b3437ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:32.804850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-44218f7c-aca8-443c-aa81-38189b3437ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:32.813036 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-44218f7c-aca8-443c-aa81-38189b3437ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149661 Jun 10 13:18:32.813595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 664/1422] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:32 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:32.840680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6bca7d7e-ab23-4b2f-8e97-65e7642d3d73 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:32.841297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6bca7d7e-ab23-4b2f-8e97-65e7642d3d73 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:32.847004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bca7d7e-ab23-4b2f-8e97-65e7642d3d73 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:32.847447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bca7d7e-ab23-4b2f-8e97-65e7642d3d73 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:32.848202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bca7d7e-ab23-4b2f-8e97-65e7642d3d73 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:32.848625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bca7d7e-ab23-4b2f-8e97-65e7642d3d73 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:32.971398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6bca7d7e-ab23-4b2f-8e97-65e7642d3d73 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:32.979275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6bca7d7e-ab23-4b2f-8e97-65e7642d3d73 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.142078 Jun 10 13:18:32.979849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 665/1423] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:32 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:33.830343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-50091ef2-661e-4302-bda9-ebf09d00f3bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:33.831004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-50091ef2-661e-4302-bda9-ebf09d00f3bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:33.837829 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-50091ef2-661e-4302-bda9-ebf09d00f3bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:33.842775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-50091ef2-661e-4302-bda9-ebf09d00f3bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:33.842775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-50091ef2-661e-4302-bda9-ebf09d00f3bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:33.842775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-50091ef2-661e-4302-bda9-ebf09d00f3bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:33.971480 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-50091ef2-661e-4302-bda9-ebf09d00f3bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:33.978976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-50091ef2-661e-4302-bda9-ebf09d00f3bd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.153245 Jun 10 13:18:33.979544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 666/1424] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:33 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:33.996130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c4a86921-f8a1-461c-8c04-b2fdf9ce7f81 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:33.996734 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c4a86921-f8a1-461c-8c04-b2fdf9ce7f81 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:34.003005 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c4a86921-f8a1-461c-8c04-b2fdf9ce7f81 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:34.003455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c4a86921-f8a1-461c-8c04-b2fdf9ce7f81 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:34.004217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c4a86921-f8a1-461c-8c04-b2fdf9ce7f81 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:34.004650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c4a86921-f8a1-461c-8c04-b2fdf9ce7f81 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:34.133020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c4a86921-f8a1-461c-8c04-b2fdf9ce7f81 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:34.141439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c4a86921-f8a1-461c-8c04-b2fdf9ce7f81 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148148 Jun 10 13:18:34.142004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 667/1425] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:33 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:34.996314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-629778fa-7c89-471c-99c8-4f0bd29249eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:34.997007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-629778fa-7c89-471c-99c8-4f0bd29249eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:35.003503 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-629778fa-7c89-471c-99c8-4f0bd29249eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:35.003939 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-629778fa-7c89-471c-99c8-4f0bd29249eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:35.004686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-629778fa-7c89-471c-99c8-4f0bd29249eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:35.005142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-629778fa-7c89-471c-99c8-4f0bd29249eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:35.129945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-629778fa-7c89-471c-99c8-4f0bd29249eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:35.137294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-629778fa-7c89-471c-99c8-4f0bd29249eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.144883 Jun 10 13:18:35.137855 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 668/1426] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:34 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:35.158862 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c5005eaa-9092-466c-868e-9482e47b41e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:35.159460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c5005eaa-9092-466c-868e-9482e47b41e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:35.166218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5005eaa-9092-466c-868e-9482e47b41e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:35.166886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5005eaa-9092-466c-868e-9482e47b41e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:35.167644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5005eaa-9092-466c-868e-9482e47b41e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:35.168068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5005eaa-9092-466c-868e-9482e47b41e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:35.295213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c5005eaa-9092-466c-868e-9482e47b41e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:35.303032 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c5005eaa-9092-466c-868e-9482e47b41e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148948 Jun 10 13:18:35.303592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 669/1427] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:35 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:36.155397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-60a4b29a-2438-4257-9ba5-257855cc0890 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:36.156078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-60a4b29a-2438-4257-9ba5-257855cc0890 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:36.162470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60a4b29a-2438-4257-9ba5-257855cc0890 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:36.162915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60a4b29a-2438-4257-9ba5-257855cc0890 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:36.163670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60a4b29a-2438-4257-9ba5-257855cc0890 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:36.164101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60a4b29a-2438-4257-9ba5-257855cc0890 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:36.289155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-60a4b29a-2438-4257-9ba5-257855cc0890 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:36.297461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-60a4b29a-2438-4257-9ba5-257855cc0890 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145211 Jun 10 13:18:36.298034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 670/1428] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:36 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:36.318978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d309b724-5e64-4f13-8bf2-dda5cc3e487e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:36.321158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d309b724-5e64-4f13-8bf2-dda5cc3e487e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:36.325999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d309b724-5e64-4f13-8bf2-dda5cc3e487e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:36.326448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d309b724-5e64-4f13-8bf2-dda5cc3e487e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:36.327215 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d309b724-5e64-4f13-8bf2-dda5cc3e487e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:36.327651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d309b724-5e64-4f13-8bf2-dda5cc3e487e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:36.455076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d309b724-5e64-4f13-8bf2-dda5cc3e487e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:36.462877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d309b724-5e64-4f13-8bf2-dda5cc3e487e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147543 Jun 10 13:18:36.463441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 671/1429] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:36 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:37.314230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a057a5f8-1ecc-4286-bbed-175c55afd8d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:37.314830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a057a5f8-1ecc-4286-bbed-175c55afd8d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:37.322354 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a057a5f8-1ecc-4286-bbed-175c55afd8d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:37.322799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a057a5f8-1ecc-4286-bbed-175c55afd8d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:37.323572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a057a5f8-1ecc-4286-bbed-175c55afd8d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:37.324728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a057a5f8-1ecc-4286-bbed-175c55afd8d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:37.456947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a057a5f8-1ecc-4286-bbed-175c55afd8d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:37.471168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a057a5f8-1ecc-4286-bbed-175c55afd8d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.160846 Jun 10 13:18:37.472115 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 672/1430] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:37 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:37.479622 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c1adb151-cd34-4e11-be7c-755b7309cf2d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:37.480230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c1adb151-cd34-4e11-be7c-755b7309cf2d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:37.483364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:18:37.490470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1adb151-cd34-4e11-be7c-755b7309cf2d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:37.497157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1adb151-cd34-4e11-be7c-755b7309cf2d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:37.497157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1adb151-cd34-4e11-be7c-755b7309cf2d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:37.497157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1adb151-cd34-4e11-be7c-755b7309cf2d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:37.621148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c1adb151-cd34-4e11-be7c-755b7309cf2d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:37.629671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c1adb151-cd34-4e11-be7c-755b7309cf2d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.153723 Jun 10 13:18:37.630239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 673/1431] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:37 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:38.488504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-10e9de7a-d445-4823-9c96-6ed9c9e25e30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:38.489175 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-10e9de7a-d445-4823-9c96-6ed9c9e25e30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:38.495514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10e9de7a-d445-4823-9c96-6ed9c9e25e30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:38.495954 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10e9de7a-d445-4823-9c96-6ed9c9e25e30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:38.496699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10e9de7a-d445-4823-9c96-6ed9c9e25e30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:38.497157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10e9de7a-d445-4823-9c96-6ed9c9e25e30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:38.626109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10e9de7a-d445-4823-9c96-6ed9c9e25e30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:38.633667 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-10e9de7a-d445-4823-9c96-6ed9c9e25e30 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149056 Jun 10 13:18:38.634218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 674/1432] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:38 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:38.646322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6bbbc553-fd73-4cdc-8fec-74b1dd5ec7e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:38.646956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6bbbc553-fd73-4cdc-8fec-74b1dd5ec7e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:38.652810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bbbc553-fd73-4cdc-8fec-74b1dd5ec7e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:38.653266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bbbc553-fd73-4cdc-8fec-74b1dd5ec7e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:38.654025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bbbc553-fd73-4cdc-8fec-74b1dd5ec7e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:38.654459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bbbc553-fd73-4cdc-8fec-74b1dd5ec7e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:38.779756 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6bbbc553-fd73-4cdc-8fec-74b1dd5ec7e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:38.787739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6bbbc553-fd73-4cdc-8fec-74b1dd5ec7e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144963 Jun 10 13:18:38.788300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 675/1433] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:38 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:39.651351 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-209a4c61-dd57-43fd-81df-8e26d1f9e0bc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:39.651972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-209a4c61-dd57-43fd-81df-8e26d1f9e0bc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:39.658930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-209a4c61-dd57-43fd-81df-8e26d1f9e0bc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:39.659369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-209a4c61-dd57-43fd-81df-8e26d1f9e0bc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:39.660117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-209a4c61-dd57-43fd-81df-8e26d1f9e0bc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:39.660544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-209a4c61-dd57-43fd-81df-8e26d1f9e0bc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:39.789033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-209a4c61-dd57-43fd-81df-8e26d1f9e0bc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:39.796459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-209a4c61-dd57-43fd-81df-8e26d1f9e0bc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149069 Jun 10 13:18:39.797053 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 676/1434] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:39 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:39.804290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8391247d-da11-41c3-943d-5d715308448e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:39.804993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8391247d-da11-41c3-943d-5d715308448e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:39.810637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8391247d-da11-41c3-943d-5d715308448e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:39.811078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8391247d-da11-41c3-943d-5d715308448e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:39.811830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8391247d-da11-41c3-943d-5d715308448e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:39.812259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8391247d-da11-41c3-943d-5d715308448e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:39.935753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8391247d-da11-41c3-943d-5d715308448e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:39.944019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8391247d-da11-41c3-943d-5d715308448e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143300 Jun 10 13:18:39.944580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 677/1435] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:39 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:40.813617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-995dacfe-a4c3-4909-879c-46580195dcc4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:40.814289 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-995dacfe-a4c3-4909-879c-46580195dcc4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:40.820608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-995dacfe-a4c3-4909-879c-46580195dcc4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:40.821082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-995dacfe-a4c3-4909-879c-46580195dcc4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:40.821830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-995dacfe-a4c3-4909-879c-46580195dcc4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:40.822257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-995dacfe-a4c3-4909-879c-46580195dcc4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:40.952753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-995dacfe-a4c3-4909-879c-46580195dcc4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:40.956453 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:18:40.961033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-995dacfe-a4c3-4909-879c-46580195dcc4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.151264 Jun 10 13:18:40.961595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 678/1436] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:40 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:40.965901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5fe28573-225f-411b-abca-4a7581e08821 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:40.966492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5fe28573-225f-411b-abca-4a7581e08821 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:40.973026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5fe28573-225f-411b-abca-4a7581e08821 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:40.973478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5fe28573-225f-411b-abca-4a7581e08821 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:40.974356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5fe28573-225f-411b-abca-4a7581e08821 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:40.974784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5fe28573-225f-411b-abca-4a7581e08821 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:41.100480 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5fe28573-225f-411b-abca-4a7581e08821 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:41.108535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5fe28573-225f-411b-abca-4a7581e08821 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146011 Jun 10 13:18:41.109120 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 679/1437] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:40 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:41.978645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-16330ebd-646b-44b0-8ec2-44e7981b2206 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:41.979312 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-16330ebd-646b-44b0-8ec2-44e7981b2206 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:41.985890 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-16330ebd-646b-44b0-8ec2-44e7981b2206 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:41.986341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-16330ebd-646b-44b0-8ec2-44e7981b2206 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:41.987071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-16330ebd-646b-44b0-8ec2-44e7981b2206 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:41.987497 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-16330ebd-646b-44b0-8ec2-44e7981b2206 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:42.113105 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-16330ebd-646b-44b0-8ec2-44e7981b2206 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:42.120449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-16330ebd-646b-44b0-8ec2-44e7981b2206 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145689 Jun 10 13:18:42.121033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 680/1438] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:41 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:42.125295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a37c4cc7-997f-4647-a8f3-27be06b6c5d8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:42.125884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a37c4cc7-997f-4647-a8f3-27be06b6c5d8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:42.131588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a37c4cc7-997f-4647-a8f3-27be06b6c5d8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:42.132066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a37c4cc7-997f-4647-a8f3-27be06b6c5d8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:42.132896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a37c4cc7-997f-4647-a8f3-27be06b6c5d8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:42.133335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a37c4cc7-997f-4647-a8f3-27be06b6c5d8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:42.260000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a37c4cc7-997f-4647-a8f3-27be06b6c5d8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:42.268580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a37c4cc7-997f-4647-a8f3-27be06b6c5d8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146554 Jun 10 13:18:42.269243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 681/1439] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:42 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:43.137550 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-12187b93-5dd9-4ac6-b3d1-5d56809f81c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:43.138228 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-12187b93-5dd9-4ac6-b3d1-5d56809f81c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:43.144657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-12187b93-5dd9-4ac6-b3d1-5d56809f81c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:43.145131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-12187b93-5dd9-4ac6-b3d1-5d56809f81c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:43.145883 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-12187b93-5dd9-4ac6-b3d1-5d56809f81c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:43.146316 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-12187b93-5dd9-4ac6-b3d1-5d56809f81c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:43.271442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-12187b93-5dd9-4ac6-b3d1-5d56809f81c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:43.278863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-12187b93-5dd9-4ac6-b3d1-5d56809f81c1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145214 Jun 10 13:18:43.279660 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 682/1440] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:43 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:43.285530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a8e87641-23e2-43b5-97a9-abcf25642f7f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:43.286145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a8e87641-23e2-43b5-97a9-abcf25642f7f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:43.291898 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8e87641-23e2-43b5-97a9-abcf25642f7f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:43.292340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8e87641-23e2-43b5-97a9-abcf25642f7f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:43.293126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8e87641-23e2-43b5-97a9-abcf25642f7f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:43.293554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8e87641-23e2-43b5-97a9-abcf25642f7f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:43.420481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a8e87641-23e2-43b5-97a9-abcf25642f7f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:43.428785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a8e87641-23e2-43b5-97a9-abcf25642f7f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147280 Jun 10 13:18:43.429359 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 683/1441] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:43 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:44.296171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d17f8c6d-246e-4a28-87ab-28a55437336d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:44.296876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d17f8c6d-246e-4a28-87ab-28a55437336d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:44.303345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d17f8c6d-246e-4a28-87ab-28a55437336d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:44.303784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d17f8c6d-246e-4a28-87ab-28a55437336d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:44.304528 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d17f8c6d-246e-4a28-87ab-28a55437336d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:44.304978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d17f8c6d-246e-4a28-87ab-28a55437336d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:44.430798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d17f8c6d-246e-4a28-87ab-28a55437336d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:44.438188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d17f8c6d-246e-4a28-87ab-28a55437336d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145894 Jun 10 13:18:44.438758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 684/1442] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:44 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:44.445007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bd9d9759-3152-4978-a93f-dab70e2ee033 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:44.445604 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bd9d9759-3152-4978-a93f-dab70e2ee033 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:44.451138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bd9d9759-3152-4978-a93f-dab70e2ee033 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:44.451582 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bd9d9759-3152-4978-a93f-dab70e2ee033 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:44.452454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bd9d9759-3152-4978-a93f-dab70e2ee033 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:44.452914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bd9d9759-3152-4978-a93f-dab70e2ee033 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:44.585302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bd9d9759-3152-4978-a93f-dab70e2ee033 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:44.593114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bd9d9759-3152-4978-a93f-dab70e2ee033 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.151457 Jun 10 13:18:44.593675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 685/1443] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:44 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:45.456480 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6ea5a54a-6988-47a9-a940-204a8f344400 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:45.457107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6ea5a54a-6988-47a9-a940-204a8f344400 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:45.463971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ea5a54a-6988-47a9-a940-204a8f344400 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:45.464419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ea5a54a-6988-47a9-a940-204a8f344400 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:45.465207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ea5a54a-6988-47a9-a940-204a8f344400 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:45.465644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ea5a54a-6988-47a9-a940-204a8f344400 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:45.595823 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6ea5a54a-6988-47a9-a940-204a8f344400 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:45.603208 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6ea5a54a-6988-47a9-a940-204a8f344400 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.151234 Jun 10 13:18:45.603778 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 686/1444] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:45 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:45.609300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-265b5c9c-5247-4510-9370-a5a32b170180 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:45.609900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-265b5c9c-5247-4510-9370-a5a32b170180 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:45.616161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-265b5c9c-5247-4510-9370-a5a32b170180 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:45.616603 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-265b5c9c-5247-4510-9370-a5a32b170180 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:45.617396 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-265b5c9c-5247-4510-9370-a5a32b170180 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:45.617828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-265b5c9c-5247-4510-9370-a5a32b170180 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:45.743376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-265b5c9c-5247-4510-9370-a5a32b170180 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:45.751290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-265b5c9c-5247-4510-9370-a5a32b170180 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145712 Jun 10 13:18:45.752452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 687/1445] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:45 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:46.619961 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d78a083e-71b2-4bde-b08c-ffad9e87325d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:46.620691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d78a083e-71b2-4bde-b08c-ffad9e87325d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:46.627083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d78a083e-71b2-4bde-b08c-ffad9e87325d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:46.627527 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d78a083e-71b2-4bde-b08c-ffad9e87325d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:46.631531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d78a083e-71b2-4bde-b08c-ffad9e87325d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:46.631531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d78a083e-71b2-4bde-b08c-ffad9e87325d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:46.764353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d78a083e-71b2-4bde-b08c-ffad9e87325d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:46.775035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d78a083e-71b2-4bde-b08c-ffad9e87325d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.158698 Jun 10 13:18:46.775968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 688/1446] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:46 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:46.782703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f785b97b-a0f7-44a3-8b87-5af592d138bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:46.783314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f785b97b-a0f7-44a3-8b87-5af592d138bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:46.790216 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f785b97b-a0f7-44a3-8b87-5af592d138bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:46.790662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f785b97b-a0f7-44a3-8b87-5af592d138bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:46.791464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f785b97b-a0f7-44a3-8b87-5af592d138bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:46.791896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f785b97b-a0f7-44a3-8b87-5af592d138bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:46.916907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f785b97b-a0f7-44a3-8b87-5af592d138bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:46.924802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f785b97b-a0f7-44a3-8b87-5af592d138bc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147415 Jun 10 13:18:46.925366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 689/1447] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:46 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:47.792482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-19187eb3-4828-471a-b7a1-7a1c33298b7c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:47.793186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-19187eb3-4828-471a-b7a1-7a1c33298b7c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:47.799553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-19187eb3-4828-471a-b7a1-7a1c33298b7c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:47.799992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-19187eb3-4828-471a-b7a1-7a1c33298b7c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:47.800774 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-19187eb3-4828-471a-b7a1-7a1c33298b7c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:47.801213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-19187eb3-4828-471a-b7a1-7a1c33298b7c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:47.929634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-19187eb3-4828-471a-b7a1-7a1c33298b7c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:47.937366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-19187eb3-4828-471a-b7a1-7a1c33298b7c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.148418 Jun 10 13:18:47.937939 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 690/1448] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:47 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:47.942299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a663474e-89df-4c1a-bd6b-0fddb82dcf67 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:47.942934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a663474e-89df-4c1a-bd6b-0fddb82dcf67 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:47.949033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a663474e-89df-4c1a-bd6b-0fddb82dcf67 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:47.949484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a663474e-89df-4c1a-bd6b-0fddb82dcf67 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:47.950236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a663474e-89df-4c1a-bd6b-0fddb82dcf67 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:47.950671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a663474e-89df-4c1a-bd6b-0fddb82dcf67 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:48.073699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a663474e-89df-4c1a-bd6b-0fddb82dcf67 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:48.082320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a663474e-89df-4c1a-bd6b-0fddb82dcf67 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143399 Jun 10 13:18:48.082879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 691/1449] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:47 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:48.953543 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e105fe3f-6ae1-4e11-9609-9ca85fa1b15c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:48.954170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e105fe3f-6ae1-4e11-9609-9ca85fa1b15c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:48.960680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e105fe3f-6ae1-4e11-9609-9ca85fa1b15c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:48.961169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e105fe3f-6ae1-4e11-9609-9ca85fa1b15c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:48.961988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e105fe3f-6ae1-4e11-9609-9ca85fa1b15c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:48.962469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e105fe3f-6ae1-4e11-9609-9ca85fa1b15c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:49.087672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e105fe3f-6ae1-4e11-9609-9ca85fa1b15c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:49.095698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e105fe3f-6ae1-4e11-9609-9ca85fa1b15c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145626 Jun 10 13:18:49.096264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 692/1450] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:48 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:49.100724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dfe45fb7-ce29-48b8-85e6-901a5febee32 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:49.101330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dfe45fb7-ce29-48b8-85e6-901a5febee32 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:49.107290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dfe45fb7-ce29-48b8-85e6-901a5febee32 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:49.107731 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dfe45fb7-ce29-48b8-85e6-901a5febee32 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:49.108485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dfe45fb7-ce29-48b8-85e6-901a5febee32 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:49.108941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dfe45fb7-ce29-48b8-85e6-901a5febee32 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:49.246807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dfe45fb7-ce29-48b8-85e6-901a5febee32 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:49.254915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dfe45fb7-ce29-48b8-85e6-901a5febee32 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.157585 Jun 10 13:18:49.255492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 693/1451] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:49 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:50.122455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1b62c77d-caaf-4210-9784-1af076640776 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:50.123101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1b62c77d-caaf-4210-9784-1af076640776 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:50.129518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b62c77d-caaf-4210-9784-1af076640776 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:50.129964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b62c77d-caaf-4210-9784-1af076640776 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:50.130713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b62c77d-caaf-4210-9784-1af076640776 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:50.131144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b62c77d-caaf-4210-9784-1af076640776 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:50.257578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1b62c77d-caaf-4210-9784-1af076640776 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:50.264975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1b62c77d-caaf-4210-9784-1af076640776 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146383 Jun 10 13:18:50.265544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 694/1452] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:50 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:50.271391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5459b765-f3fa-4a67-9a88-9d474c2fa5d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:50.271986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5459b765-f3fa-4a67-9a88-9d474c2fa5d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:50.277679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5459b765-f3fa-4a67-9a88-9d474c2fa5d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:50.278125 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5459b765-f3fa-4a67-9a88-9d474c2fa5d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:50.278880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5459b765-f3fa-4a67-9a88-9d474c2fa5d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:50.279314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5459b765-f3fa-4a67-9a88-9d474c2fa5d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:50.296872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:18:50.408219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5459b765-f3fa-4a67-9a88-9d474c2fa5d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:50.416006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5459b765-f3fa-4a67-9a88-9d474c2fa5d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148115 Jun 10 13:18:50.416565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 695/1453] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:50 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:51.283221 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a86790a9-846b-42d3-86b6-37e0d4d82be3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:51.283821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a86790a9-846b-42d3-86b6-37e0d4d82be3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:51.290218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a86790a9-846b-42d3-86b6-37e0d4d82be3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:51.290657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a86790a9-846b-42d3-86b6-37e0d4d82be3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:51.291400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a86790a9-846b-42d3-86b6-37e0d4d82be3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:51.291823 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a86790a9-846b-42d3-86b6-37e0d4d82be3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:51.417177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a86790a9-846b-42d3-86b6-37e0d4d82be3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:51.425148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a86790a9-846b-42d3-86b6-37e0d4d82be3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146213 Jun 10 13:18:51.425705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 696/1454] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:51 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:51.432411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-45047d9f-2423-487c-ae51-888d8c0f2ef4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:51.433038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-45047d9f-2423-487c-ae51-888d8c0f2ef4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:51.438705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45047d9f-2423-487c-ae51-888d8c0f2ef4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:51.439149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45047d9f-2423-487c-ae51-888d8c0f2ef4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:51.439897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45047d9f-2423-487c-ae51-888d8c0f2ef4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:51.440326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45047d9f-2423-487c-ae51-888d8c0f2ef4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:51.566085 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-45047d9f-2423-487c-ae51-888d8c0f2ef4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:51.574066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-45047d9f-2423-487c-ae51-888d8c0f2ef4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145195 Jun 10 13:18:51.574630 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 697/1455] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:51 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:52.442593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7fac7444-1b73-4700-b51b-3994bd864efc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:52.443218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7fac7444-1b73-4700-b51b-3994bd864efc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:52.450254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7fac7444-1b73-4700-b51b-3994bd864efc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:52.450708 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7fac7444-1b73-4700-b51b-3994bd864efc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:52.451487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7fac7444-1b73-4700-b51b-3994bd864efc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:52.451920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7fac7444-1b73-4700-b51b-3994bd864efc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:52.590773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7fac7444-1b73-4700-b51b-3994bd864efc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:52.598179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7fac7444-1b73-4700-b51b-3994bd864efc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.160115 Jun 10 13:18:52.598745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 698/1456] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:52 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:52.603004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cc4dc3df-f21f-418b-8988-06dc417d3f5f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:52.603598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cc4dc3df-f21f-418b-8988-06dc417d3f5f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:52.609253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cc4dc3df-f21f-418b-8988-06dc417d3f5f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:52.609697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cc4dc3df-f21f-418b-8988-06dc417d3f5f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:52.610946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cc4dc3df-f21f-418b-8988-06dc417d3f5f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:52.611387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cc4dc3df-f21f-418b-8988-06dc417d3f5f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:52.732887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cc4dc3df-f21f-418b-8988-06dc417d3f5f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:52.741542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cc4dc3df-f21f-418b-8988-06dc417d3f5f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.141804 Jun 10 13:18:52.742115 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 699/1457] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:52 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:53.615910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-746df83e-1501-4091-a4dc-688f7ff7c1cc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:53.616529 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-746df83e-1501-4091-a4dc-688f7ff7c1cc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:53.622988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-746df83e-1501-4091-a4dc-688f7ff7c1cc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:53.623431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-746df83e-1501-4091-a4dc-688f7ff7c1cc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:53.624187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-746df83e-1501-4091-a4dc-688f7ff7c1cc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:53.624613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-746df83e-1501-4091-a4dc-688f7ff7c1cc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:53.753645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-746df83e-1501-4091-a4dc-688f7ff7c1cc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:53.761161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-746df83e-1501-4091-a4dc-688f7ff7c1cc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149123 Jun 10 13:18:53.761730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 700/1458] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:53 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:53.766083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-240a8a72-7d44-47c9-b526-5c150075997f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:53.766679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-240a8a72-7d44-47c9-b526-5c150075997f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:53.772692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-240a8a72-7d44-47c9-b526-5c150075997f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:53.773173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-240a8a72-7d44-47c9-b526-5c150075997f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:53.773930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-240a8a72-7d44-47c9-b526-5c150075997f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:53.774364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-240a8a72-7d44-47c9-b526-5c150075997f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:53.898142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-240a8a72-7d44-47c9-b526-5c150075997f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:53.906035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-240a8a72-7d44-47c9-b526-5c150075997f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143362 Jun 10 13:18:53.906612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 701/1459] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:53 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:54.778667 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c48ad402-976b-46b3-90f7-5c8c524fa16e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:54.779333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c48ad402-976b-46b3-90f7-5c8c524fa16e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:54.786300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c48ad402-976b-46b3-90f7-5c8c524fa16e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:54.786744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c48ad402-976b-46b3-90f7-5c8c524fa16e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:54.787488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c48ad402-976b-46b3-90f7-5c8c524fa16e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:54.787916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c48ad402-976b-46b3-90f7-5c8c524fa16e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:54.916203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c48ad402-976b-46b3-90f7-5c8c524fa16e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:54.923584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c48ad402-976b-46b3-90f7-5c8c524fa16e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.148753 Jun 10 13:18:54.924144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 702/1460] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:54 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:54.928391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-37d1547d-ddbc-4d90-8b71-24f04bc45830 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:54.929009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-37d1547d-ddbc-4d90-8b71-24f04bc45830 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:54.934397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-37d1547d-ddbc-4d90-8b71-24f04bc45830 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:54.934836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-37d1547d-ddbc-4d90-8b71-24f04bc45830 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:54.935581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-37d1547d-ddbc-4d90-8b71-24f04bc45830 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:54.936006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-37d1547d-ddbc-4d90-8b71-24f04bc45830 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:55.059851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-37d1547d-ddbc-4d90-8b71-24f04bc45830 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:55.068058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-37d1547d-ddbc-4d90-8b71-24f04bc45830 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.142989 Jun 10 13:18:55.068616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 703/1461] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:54 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:55.940287 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-263247de-e569-4f36-b796-452b62ac6273 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:55.940919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-263247de-e569-4f36-b796-452b62ac6273 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:55.947357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-263247de-e569-4f36-b796-452b62ac6273 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:55.947803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-263247de-e569-4f36-b796-452b62ac6273 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:55.948552 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-263247de-e569-4f36-b796-452b62ac6273 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:55.949007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-263247de-e569-4f36-b796-452b62ac6273 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:56.073452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-263247de-e569-4f36-b796-452b62ac6273 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:56.081385 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-263247de-e569-4f36-b796-452b62ac6273 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145110 Jun 10 13:18:56.081962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 704/1462] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:55 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:56.086341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-75af1e43-5eda-4a0c-8a68-44bc1dcab527 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:56.086943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-75af1e43-5eda-4a0c-8a68-44bc1dcab527 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:56.097641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75af1e43-5eda-4a0c-8a68-44bc1dcab527 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:56.098323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75af1e43-5eda-4a0c-8a68-44bc1dcab527 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:56.099151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75af1e43-5eda-4a0c-8a68-44bc1dcab527 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:56.099587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75af1e43-5eda-4a0c-8a68-44bc1dcab527 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:56.226000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-75af1e43-5eda-4a0c-8a68-44bc1dcab527 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:56.234150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-75af1e43-5eda-4a0c-8a68-44bc1dcab527 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.151254 Jun 10 13:18:56.234716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 705/1463] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:56 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:18:57.098429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-97684d11-8319-441d-b92e-a948c40acd34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:57.099034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-97684d11-8319-441d-b92e-a948c40acd34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:57.105496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97684d11-8319-441d-b92e-a948c40acd34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:57.105929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97684d11-8319-441d-b92e-a948c40acd34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:57.106953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97684d11-8319-441d-b92e-a948c40acd34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:57.107383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97684d11-8319-441d-b92e-a948c40acd34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:57.234995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-97684d11-8319-441d-b92e-a948c40acd34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:57.243081 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-97684d11-8319-441d-b92e-a948c40acd34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.148643 Jun 10 13:18:57.243644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 706/1464] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:57 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:57.250615 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0a6f981b-91f2-4d66-9fb7-7018c2982f9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:57.251214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0a6f981b-91f2-4d66-9fb7-7018c2982f9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:57.257127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0a6f981b-91f2-4d66-9fb7-7018c2982f9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:57.257645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0a6f981b-91f2-4d66-9fb7-7018c2982f9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:57.258440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0a6f981b-91f2-4d66-9fb7-7018c2982f9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:57.258867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0a6f981b-91f2-4d66-9fb7-7018c2982f9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:57.387346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0a6f981b-91f2-4d66-9fb7-7018c2982f9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:57.395266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0a6f981b-91f2-4d66-9fb7-7018c2982f9a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148450 Jun 10 13:18:57.395830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 707/1465] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:57 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:58.259500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e74118a8-ab75-4c43-abb1-e2ce055364c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:58.260144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e74118a8-ab75-4c43-abb1-e2ce055364c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:58.266603 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e74118a8-ab75-4c43-abb1-e2ce055364c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:58.267050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e74118a8-ab75-4c43-abb1-e2ce055364c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:58.267806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e74118a8-ab75-4c43-abb1-e2ce055364c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:58.268236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e74118a8-ab75-4c43-abb1-e2ce055364c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:58.393457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e74118a8-ab75-4c43-abb1-e2ce055364c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:58.400755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e74118a8-ab75-4c43-abb1-e2ce055364c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145157 Jun 10 13:18:58.401342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 708/1466] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:58 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:58.411282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0f8d0f21-fc9a-4997-8dbb-2d82ac173691 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:58.411885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0f8d0f21-fc9a-4997-8dbb-2d82ac173691 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:58.417657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f8d0f21-fc9a-4997-8dbb-2d82ac173691 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:58.418101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f8d0f21-fc9a-4997-8dbb-2d82ac173691 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:58.418902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f8d0f21-fc9a-4997-8dbb-2d82ac173691 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:58.419283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f8d0f21-fc9a-4997-8dbb-2d82ac173691 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:58.543873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0f8d0f21-fc9a-4997-8dbb-2d82ac173691 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:58.552214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0f8d0f21-fc9a-4997-8dbb-2d82ac173691 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144479 Jun 10 13:18:58.552872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 709/1467] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:58 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:59.417741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-662b9a82-ac21-4dc7-b02f-c17f62a80a85 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:59.418370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-662b9a82-ac21-4dc7-b02f-c17f62a80a85 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:59.424873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-662b9a82-ac21-4dc7-b02f-c17f62a80a85 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:59.425308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-662b9a82-ac21-4dc7-b02f-c17f62a80a85 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:59.426049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-662b9a82-ac21-4dc7-b02f-c17f62a80a85 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:59.426469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-662b9a82-ac21-4dc7-b02f-c17f62a80a85 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:59.551025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-662b9a82-ac21-4dc7-b02f-c17f62a80a85 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:59.558558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-662b9a82-ac21-4dc7-b02f-c17f62a80a85 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.144637 Jun 10 13:18:59.559144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 710/1468] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:59 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:18:59.569293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-49104d34-5097-4c7c-9939-b3ef656998a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:18:59.569893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-49104d34-5097-4c7c-9939-b3ef656998a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:18:59.575849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-49104d34-5097-4c7c-9939-b3ef656998a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:59.576286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-49104d34-5097-4c7c-9939-b3ef656998a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:59.577068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-49104d34-5097-4c7c-9939-b3ef656998a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:18:59.577502 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-49104d34-5097-4c7c-9939-b3ef656998a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:18:59.702350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-49104d34-5097-4c7c-9939-b3ef656998a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:18:59.710885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-49104d34-5097-4c7c-9939-b3ef656998a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145146 Jun 10 13:18:59.713442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 711/1469] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:18:59 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:00.575904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9653e632-6a7f-4f4f-a37a-44063f87de4d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:00.576535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9653e632-6a7f-4f4f-a37a-44063f87de4d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:00.583784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9653e632-6a7f-4f4f-a37a-44063f87de4d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:00.584225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9653e632-6a7f-4f4f-a37a-44063f87de4d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:00.585012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9653e632-6a7f-4f4f-a37a-44063f87de4d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:00.585447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9653e632-6a7f-4f4f-a37a-44063f87de4d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:00.589464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:19:00.713955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9653e632-6a7f-4f4f-a37a-44063f87de4d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:00.721897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9653e632-6a7f-4f4f-a37a-44063f87de4d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149865 Jun 10 13:19:00.722455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 712/1470] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:00 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:00.727491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-313f1748-5212-4d1d-a8e3-7192c05e5bc1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:00.728246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-313f1748-5212-4d1d-a8e3-7192c05e5bc1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:00.734245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-313f1748-5212-4d1d-a8e3-7192c05e5bc1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:00.734700 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-313f1748-5212-4d1d-a8e3-7192c05e5bc1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:00.735452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-313f1748-5212-4d1d-a8e3-7192c05e5bc1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:00.735885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-313f1748-5212-4d1d-a8e3-7192c05e5bc1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:00.858749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-313f1748-5212-4d1d-a8e3-7192c05e5bc1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:00.866731 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-313f1748-5212-4d1d-a8e3-7192c05e5bc1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.142719 Jun 10 13:19:00.867306 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 713/1471] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:00 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:01.739369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3d94856b-13ca-4afe-a922-732ace32b08b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:01.740054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3d94856b-13ca-4afe-a922-732ace32b08b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:01.746029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3d94856b-13ca-4afe-a922-732ace32b08b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:01.746475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3d94856b-13ca-4afe-a922-732ace32b08b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:01.747232 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3d94856b-13ca-4afe-a922-732ace32b08b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:01.747663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3d94856b-13ca-4afe-a922-732ace32b08b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:01.888032 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3d94856b-13ca-4afe-a922-732ace32b08b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:01.896103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3d94856b-13ca-4afe-a922-732ace32b08b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.160757 Jun 10 13:19:01.896662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 714/1472] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:01 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:01.901080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3891d409-dd01-4f7d-9067-8da6f788a0fb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:01.901668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3891d409-dd01-4f7d-9067-8da6f788a0fb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:01.907042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3891d409-dd01-4f7d-9067-8da6f788a0fb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:01.907482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3891d409-dd01-4f7d-9067-8da6f788a0fb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:01.908271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3891d409-dd01-4f7d-9067-8da6f788a0fb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:01.908701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3891d409-dd01-4f7d-9067-8da6f788a0fb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:02.031299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3891d409-dd01-4f7d-9067-8da6f788a0fb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:02.039104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3891d409-dd01-4f7d-9067-8da6f788a0fb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.141429 Jun 10 13:19:02.039668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 715/1473] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:01 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:19:02.914014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cfac7519-d63d-4b28-b1bc-61e504fad8f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:02.914689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cfac7519-d63d-4b28-b1bc-61e504fad8f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:02.921137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfac7519-d63d-4b28-b1bc-61e504fad8f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:02.921582 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfac7519-d63d-4b28-b1bc-61e504fad8f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:02.922332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfac7519-d63d-4b28-b1bc-61e504fad8f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:02.922765 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfac7519-d63d-4b28-b1bc-61e504fad8f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:03.047811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cfac7519-d63d-4b28-b1bc-61e504fad8f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:03.055355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cfac7519-d63d-4b28-b1bc-61e504fad8f6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145571 Jun 10 13:19:03.055918 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 716/1474] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:02 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:03.060248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f94dba89-162f-4d49-98f8-2a6b1508d0ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:03.060873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f94dba89-162f-4d49-98f8-2a6b1508d0ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:03.066624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f94dba89-162f-4d49-98f8-2a6b1508d0ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:03.067086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f94dba89-162f-4d49-98f8-2a6b1508d0ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:03.067870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f94dba89-162f-4d49-98f8-2a6b1508d0ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:03.068295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f94dba89-162f-4d49-98f8-2a6b1508d0ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:03.196083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f94dba89-162f-4d49-98f8-2a6b1508d0ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:03.203689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f94dba89-162f-4d49-98f8-2a6b1508d0ab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146836 Jun 10 13:19:03.204247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 717/1475] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:03 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:19:04.072555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8a82e50b-9517-46bc-a4f8-6b98c9a40dfb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:04.073250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8a82e50b-9517-46bc-a4f8-6b98c9a40dfb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:04.079624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a82e50b-9517-46bc-a4f8-6b98c9a40dfb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:04.080705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a82e50b-9517-46bc-a4f8-6b98c9a40dfb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:04.081482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a82e50b-9517-46bc-a4f8-6b98c9a40dfb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:04.081909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a82e50b-9517-46bc-a4f8-6b98c9a40dfb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:04.210818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8a82e50b-9517-46bc-a4f8-6b98c9a40dfb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:04.218394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8a82e50b-9517-46bc-a4f8-6b98c9a40dfb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149631 Jun 10 13:19:04.218962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 718/1476] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:04 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:04.223433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-217ef966-e0fa-4db9-b724-f4a4b69dd74d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:04.224037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-217ef966-e0fa-4db9-b724-f4a4b69dd74d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:04.229816 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-217ef966-e0fa-4db9-b724-f4a4b69dd74d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:04.230262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-217ef966-e0fa-4db9-b724-f4a4b69dd74d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:04.231337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-217ef966-e0fa-4db9-b724-f4a4b69dd74d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:04.231767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-217ef966-e0fa-4db9-b724-f4a4b69dd74d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:04.357800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-217ef966-e0fa-4db9-b724-f4a4b69dd74d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:04.366362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-217ef966-e0fa-4db9-b724-f4a4b69dd74d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146437 Jun 10 13:19:04.366931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 719/1477] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:04 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:19:05.235638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-19271255-1b5c-422f-9a4c-5024ff77f91a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:05.236340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-19271255-1b5c-422f-9a4c-5024ff77f91a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:05.242737 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-19271255-1b5c-422f-9a4c-5024ff77f91a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:05.243181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-19271255-1b5c-422f-9a4c-5024ff77f91a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:05.243937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-19271255-1b5c-422f-9a4c-5024ff77f91a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:05.244370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-19271255-1b5c-422f-9a4c-5024ff77f91a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:05.370986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-19271255-1b5c-422f-9a4c-5024ff77f91a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:05.378779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-19271255-1b5c-422f-9a4c-5024ff77f91a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146985 Jun 10 13:19:05.379336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 720/1478] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:05 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:05.383656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0b37b7d7-2ba7-4832-be9f-e2ef94fd0a94 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:05.384253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0b37b7d7-2ba7-4832-be9f-e2ef94fd0a94 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:05.389887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b37b7d7-2ba7-4832-be9f-e2ef94fd0a94 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:05.390338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b37b7d7-2ba7-4832-be9f-e2ef94fd0a94 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:05.391097 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b37b7d7-2ba7-4832-be9f-e2ef94fd0a94 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:05.391535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b37b7d7-2ba7-4832-be9f-e2ef94fd0a94 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:05.516352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0b37b7d7-2ba7-4832-be9f-e2ef94fd0a94 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:05.524755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0b37b7d7-2ba7-4832-be9f-e2ef94fd0a94 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144490 Jun 10 13:19:05.525332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 721/1479] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:05 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:19:06.395531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d5e59df7-e528-4106-92ed-70aa176b2ee5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:06.396125 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d5e59df7-e528-4106-92ed-70aa176b2ee5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:06.402793 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5e59df7-e528-4106-92ed-70aa176b2ee5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:06.403237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5e59df7-e528-4106-92ed-70aa176b2ee5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:06.404001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5e59df7-e528-4106-92ed-70aa176b2ee5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:06.404436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5e59df7-e528-4106-92ed-70aa176b2ee5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:06.529027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d5e59df7-e528-4106-92ed-70aa176b2ee5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:06.537222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d5e59df7-e528-4106-92ed-70aa176b2ee5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145481 Jun 10 13:19:06.537786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 722/1480] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:06 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:06.542106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1d43e703-94fe-45fb-a84a-3560accda7c7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:06.542708 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1d43e703-94fe-45fb-a84a-3560accda7c7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:06.545730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:19:06.549114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d43e703-94fe-45fb-a84a-3560accda7c7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:06.552449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d43e703-94fe-45fb-a84a-3560accda7c7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:06.552449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d43e703-94fe-45fb-a84a-3560accda7c7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:06.552449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d43e703-94fe-45fb-a84a-3560accda7c7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:06.675556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1d43e703-94fe-45fb-a84a-3560accda7c7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:06.683263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1d43e703-94fe-45fb-a84a-3560accda7c7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144545 Jun 10 13:19:06.683824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 723/1481] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:06 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:19:07.554862 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d249e061-f2b8-4539-8b70-68a5d8828d56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:07.555548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d249e061-f2b8-4539-8b70-68a5d8828d56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:07.562004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d249e061-f2b8-4539-8b70-68a5d8828d56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:07.562394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d249e061-f2b8-4539-8b70-68a5d8828d56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:07.563148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d249e061-f2b8-4539-8b70-68a5d8828d56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:07.563577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d249e061-f2b8-4539-8b70-68a5d8828d56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:07.694447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d249e061-f2b8-4539-8b70-68a5d8828d56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:07.701943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d249e061-f2b8-4539-8b70-68a5d8828d56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.150937 Jun 10 13:19:07.702503 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 724/1482] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:07 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:07.706945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c47cc4e0-479e-4b7e-bb6f-bbc05fcf12cc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:07.707544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c47cc4e0-479e-4b7e-bb6f-bbc05fcf12cc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:07.714884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c47cc4e0-479e-4b7e-bb6f-bbc05fcf12cc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:07.715331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c47cc4e0-479e-4b7e-bb6f-bbc05fcf12cc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:07.716112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c47cc4e0-479e-4b7e-bb6f-bbc05fcf12cc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:07.716541 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c47cc4e0-479e-4b7e-bb6f-bbc05fcf12cc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:07.846448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c47cc4e0-479e-4b7e-bb6f-bbc05fcf12cc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:07.854824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c47cc4e0-479e-4b7e-bb6f-bbc05fcf12cc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.151282 Jun 10 13:19:07.855395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 725/1483] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:07 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:19:08.720372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8f46d455-9d8c-4c22-9601-d32b370f1c6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:08.721015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8f46d455-9d8c-4c22-9601-d32b370f1c6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:08.727608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f46d455-9d8c-4c22-9601-d32b370f1c6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:08.728088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f46d455-9d8c-4c22-9601-d32b370f1c6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:08.728871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f46d455-9d8c-4c22-9601-d32b370f1c6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:08.729300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f46d455-9d8c-4c22-9601-d32b370f1c6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:08.861646 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8f46d455-9d8c-4c22-9601-d32b370f1c6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:08.869554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8f46d455-9d8c-4c22-9601-d32b370f1c6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.153033 Jun 10 13:19:08.870111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 726/1484] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:08 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:08.875236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6ed824dd-13e0-4adf-b961-f07c1f44774a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:08.875836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6ed824dd-13e0-4adf-b961-f07c1f44774a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:08.882111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ed824dd-13e0-4adf-b961-f07c1f44774a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:08.882555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ed824dd-13e0-4adf-b961-f07c1f44774a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:08.883311 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ed824dd-13e0-4adf-b961-f07c1f44774a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:08.883741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ed824dd-13e0-4adf-b961-f07c1f44774a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:09.009199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6ed824dd-13e0-4adf-b961-f07c1f44774a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:09.016996 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6ed824dd-13e0-4adf-b961-f07c1f44774a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145202 Jun 10 13:19:09.017561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 727/1485] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:08 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:19:09.887640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4794a1af-7d57-4fc3-acbd-91016ca3822e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:09.888285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4794a1af-7d57-4fc3-acbd-91016ca3822e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:09.895387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4794a1af-7d57-4fc3-acbd-91016ca3822e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:09.895828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4794a1af-7d57-4fc3-acbd-91016ca3822e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:09.896581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4794a1af-7d57-4fc3-acbd-91016ca3822e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:09.897060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4794a1af-7d57-4fc3-acbd-91016ca3822e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:10.021230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4794a1af-7d57-4fc3-acbd-91016ca3822e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:10.024597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:19:10.028893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4794a1af-7d57-4fc3-acbd-91016ca3822e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145057 Jun 10 13:19:10.029453 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 728/1486] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:09 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:10.033781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1d10de1e-5bfc-4d5f-b4db-744401c3c89e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:10.042262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1d10de1e-5bfc-4d5f-b4db-744401c3c89e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:10.042262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d10de1e-5bfc-4d5f-b4db-744401c3c89e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:10.042262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d10de1e-5bfc-4d5f-b4db-744401c3c89e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:10.042262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d10de1e-5bfc-4d5f-b4db-744401c3c89e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:10.042669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d10de1e-5bfc-4d5f-b4db-744401c3c89e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:10.167622 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1d10de1e-5bfc-4d5f-b4db-744401c3c89e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:10.175603 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1d10de1e-5bfc-4d5f-b4db-744401c3c89e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145144 Jun 10 13:19:10.176326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 729/1487] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:10 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:19:11.045780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a2c07141-5ed5-4eee-892f-ffd30c77e72c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:11.046459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a2c07141-5ed5-4eee-892f-ffd30c77e72c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:11.052937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2c07141-5ed5-4eee-892f-ffd30c77e72c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:11.055765 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2c07141-5ed5-4eee-892f-ffd30c77e72c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:11.055765 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2c07141-5ed5-4eee-892f-ffd30c77e72c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:11.055765 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2c07141-5ed5-4eee-892f-ffd30c77e72c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:11.178820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a2c07141-5ed5-4eee-892f-ffd30c77e72c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:11.186204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a2c07141-5ed5-4eee-892f-ffd30c77e72c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.144567 Jun 10 13:19:11.186767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 730/1488] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:11 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:11.191057 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7de795d5-60f8-4f92-af07-4fa332ae6bd5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:11.191653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7de795d5-60f8-4f92-af07-4fa332ae6bd5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:11.197369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7de795d5-60f8-4f92-af07-4fa332ae6bd5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:11.197832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7de795d5-60f8-4f92-af07-4fa332ae6bd5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:11.198561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7de795d5-60f8-4f92-af07-4fa332ae6bd5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:11.198992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7de795d5-60f8-4f92-af07-4fa332ae6bd5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:11.497881 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7de795d5-60f8-4f92-af07-4fa332ae6bd5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:11.505865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7de795d5-60f8-4f92-af07-4fa332ae6bd5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.318139 Jun 10 13:19:11.506434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 731/1489] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:11 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:19:12.203321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e7f51bc1-fe63-4a77-baac-58b7d2b42979 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:12.203927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e7f51bc1-fe63-4a77-baac-58b7d2b42979 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:12.211104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7f51bc1-fe63-4a77-baac-58b7d2b42979 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:12.211551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7f51bc1-fe63-4a77-baac-58b7d2b42979 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:12.212313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7f51bc1-fe63-4a77-baac-58b7d2b42979 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:12.212744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7f51bc1-fe63-4a77-baac-58b7d2b42979 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:12.338979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e7f51bc1-fe63-4a77-baac-58b7d2b42979 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:12.346685 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e7f51bc1-fe63-4a77-baac-58b7d2b42979 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.147220 Jun 10 13:19:12.347241 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 732/1490] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:12 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:12.522694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9c317490-df3e-4982-8521-6bb925687752 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:12.523303 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9c317490-df3e-4982-8521-6bb925687752 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:12.529211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c317490-df3e-4982-8521-6bb925687752 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:12.529661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c317490-df3e-4982-8521-6bb925687752 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:12.530419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c317490-df3e-4982-8521-6bb925687752 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:12.530852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c317490-df3e-4982-8521-6bb925687752 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:12.656706 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9c317490-df3e-4982-8521-6bb925687752 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:12.665194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9c317490-df3e-4982-8521-6bb925687752 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146141 Jun 10 13:19:12.665762 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 733/1491] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:12 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:13.363889 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a2372ee3-95c1-445a-86bf-038e743bea0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:13.364505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a2372ee3-95c1-445a-86bf-038e743bea0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:13.371055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2372ee3-95c1-445a-86bf-038e743bea0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:13.371505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2372ee3-95c1-445a-86bf-038e743bea0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:13.372263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2372ee3-95c1-445a-86bf-038e743bea0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:13.372697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2372ee3-95c1-445a-86bf-038e743bea0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:13.496971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a2372ee3-95c1-445a-86bf-038e743bea0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:13.505083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a2372ee3-95c1-445a-86bf-038e743bea0d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145569 Jun 10 13:19:13.505657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 734/1492] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:13 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:13.681994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1c61b28e-b601-4690-b79d-37f3aa54d06b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:13.682594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1c61b28e-b601-4690-b79d-37f3aa54d06b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:13.688400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c61b28e-b601-4690-b79d-37f3aa54d06b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:13.688872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c61b28e-b601-4690-b79d-37f3aa54d06b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:13.689622 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c61b28e-b601-4690-b79d-37f3aa54d06b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:13.690048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c61b28e-b601-4690-b79d-37f3aa54d06b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:13.813248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1c61b28e-b601-4690-b79d-37f3aa54d06b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:13.821109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1c61b28e-b601-4690-b79d-37f3aa54d06b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.142801 Jun 10 13:19:13.821676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 735/1493] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:13 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:14.522068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c255e4be-4761-4bd7-991d-8f1cc5fc994a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:14.522680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c255e4be-4761-4bd7-991d-8f1cc5fc994a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:14.529068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c255e4be-4761-4bd7-991d-8f1cc5fc994a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:14.529511 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c255e4be-4761-4bd7-991d-8f1cc5fc994a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:14.530256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c255e4be-4761-4bd7-991d-8f1cc5fc994a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:14.530680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c255e4be-4761-4bd7-991d-8f1cc5fc994a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:14.660956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c255e4be-4761-4bd7-991d-8f1cc5fc994a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:14.668357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c255e4be-4761-4bd7-991d-8f1cc5fc994a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.150178 Jun 10 13:19:14.668944 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 736/1494] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:14 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:14.837661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-45c31a6a-9157-4ef6-a810-80f8d8d6664b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:14.838213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-45c31a6a-9157-4ef6-a810-80f8d8d6664b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:14.844136 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45c31a6a-9157-4ef6-a810-80f8d8d6664b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:14.844577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45c31a6a-9157-4ef6-a810-80f8d8d6664b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:14.845359 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45c31a6a-9157-4ef6-a810-80f8d8d6664b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:14.845794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45c31a6a-9157-4ef6-a810-80f8d8d6664b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:14.970011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-45c31a6a-9157-4ef6-a810-80f8d8d6664b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:19:14.977996 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-45c31a6a-9157-4ef6-a810-80f8d8d6664b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144030 Jun 10 13:19:14.978566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 737/1495] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:19:14 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:19:15.294459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-03541bab-8ff7-4826-9641-b3350ed1c65f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:19:15.294945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-03541bab-8ff7-4826-9641-b3350ed1c65f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:19:15.301758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-03541bab-8ff7-4826-9641-b3350ed1c65f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:15.302254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-03541bab-8ff7-4826-9641-b3350ed1c65f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:15.303043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-03541bab-8ff7-4826-9641-b3350ed1c65f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:19:15.303472 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-03541bab-8ff7-4826-9641-b3350ed1c65f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:19:18.245422 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-6773613a-6114-437b-935e-0e3e8bece4be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lazy-loading 'services' on Instance uuid a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:19:18.575860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-03541bab-8ff7-4826-9641-b3350ed1c65f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lazy-loading 'services' on Instance uuid a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:19:25.298740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=30 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:19:28.249538 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=4 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:19:28.579809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=62 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:19:44.353791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:19:44.354338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:19:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e (ip 10.222.0.25) !!! Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.wsgi [None req-6773613a-6114-437b-935e-0e3e8bece4be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dfb22d6f-2b6c-4e2e-be5c-741b82d79fad) Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6773613a-6114-437b-935e-0e3e8bece4be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Request method failure captured: Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: request: DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e HTTP/1.1 Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept: application/json Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept-Encoding: identity Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Connection: close Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Length: 0 Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Type: application/json Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Host: 10.222.0.25 Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Openstack-System-Scope: None Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: User-Agent: python-urllib3/1.26.9 Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Auth-Token: gAAAAABio0HlyICQX9l5KFYyUsFzMtUgTAzPRfxQzNN7dM9kV13g0iWQTn9QMENYtfZqE20ETOnCUUxvj1yudvSxd1bFTvMSIvc_eVAfoGvb2Phk1I--eUGA8-ePhD-Rp01rKEhzSDpYm-ppUB1HJXel5mUDpZwxED5NTdU7SM0nV7mlEGF1wbo Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Id: None Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Name: None Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Identity-Status: Confirmed Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Is-Admin-Project: True Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Id: default Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Name: Default Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Id: ff2616337ae440a89047fd9a50218387 Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Role: reader,member Jun 10 13:20:15.904110 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Roles: reader,member Jun 10 13:20:15.906238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ff2616337ae440a89047fd9a50218387"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ff2616337ae440a89047fd9a50218387"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ff2616337ae440a89047fd9a50218387", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ff2616337ae440a89047fd9a50218387"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:20:15.906238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:20:15.906238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Id: ff2616337ae440a89047fd9a50218387 Jun 10 13:20:15.906238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:20:15.906238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User: tempest-VolumesSnapshotTestJSON-1698707508-project Jun 10 13:20:15.906238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Id: default Jun 10 13:20:15.906238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Name: Default Jun 10 13:20:15.906238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Id: 849e36f8653d4d1f9a7fb1df8fb913a9 Jun 10 13:20:15.906238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Name: tempest-VolumesSnapshotTestJSON-1698707508-project Jun 10 13:20:15.906238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: method: > Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dfb22d6f-2b6c-4e2e-be5c-741b82d79fad) Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: action_args: {'server_id': 'a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744', 'id': '802588c0-10d5-4ca0-8cbb-585dee61c94e'} Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jun 10 13:20:15.907037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dfb22d6f-2b6c-4e2e-be5c-741b82d79fad) Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:20:15.908248 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dfb22d6f-2b6c-4e2e-be5c-741b82d79fad) Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:20:15.910038 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dfb22d6f-2b6c-4e2e-be5c-741b82d79fad) Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:20:15.911300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dfb22d6f-2b6c-4e2e-be5c-741b82d79fad) Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dfb22d6f-2b6c-4e2e-be5c-741b82d79fad) Jun 10 13:20:15.912600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.913945 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6773613a-6114-437b-935e-0e3e8bece4be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dfb22d6f-2b6c-4e2e-be5c-741b82d79fad) {{(pid=118093) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:20:15.913945 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6773613a-6114-437b-935e-0e3e8bece4be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e" status: 400 len: 269 microversion: 2.1 time: 120.012138 Jun 10 13:20:15.913945 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 759/1496] 10.222.0.25 () {58 vars in 1415 bytes} [Fri Jun 10 13:18:15 2022] DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e => generated 269 bytes in 120013 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jun 10 13:20:15.913945 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-778e4a69-c5c2-4b86-9471-b34002410e07 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:15.914538 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-778e4a69-c5c2-4b86-9471-b34002410e07 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:15.917378 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-778e4a69-c5c2-4b86-9471-b34002410e07 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:15.917830 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-778e4a69-c5c2-4b86-9471-b34002410e07 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:15.918603 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-778e4a69-c5c2-4b86-9471-b34002410e07 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:15.919045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-778e4a69-c5c2-4b86-9471-b34002410e07 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:15.977022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-03541bab-8ff7-4826-9641-b3350ed1c65f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6679e3fc-124e-4196-a5e6-0825363a9ac8) Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-03541bab-8ff7-4826-9641-b3350ed1c65f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Request method failure captured: Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e HTTP/1.1 Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0HlyICQX9l5KFYyUsFzMtUgTAzPRfxQzNN7dM9kV13g0iWQTn9QMENYtfZqE20ETOnCUUxvj1yudvSxd1bFTvMSIvc_eVAfoGvb2Phk1I--eUGA8-ePhD-Rp01rKEhzSDpYm-ppUB1HJXel5mUDpZwxED5NTdU7SM0nV7mlEGF1wbo Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: ff2616337ae440a89047fd9a50218387 Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:20:15.982803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:20:15.984285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ff2616337ae440a89047fd9a50218387"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ff2616337ae440a89047fd9a50218387"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ff2616337ae440a89047fd9a50218387", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ff2616337ae440a89047fd9a50218387"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:20:15.984285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:20:15.984285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: ff2616337ae440a89047fd9a50218387 Jun 10 13:20:15.984285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:20:15.984285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesSnapshotTestJSON-1698707508-project Jun 10 13:20:15.984285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:20:15.984285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:20:15.984285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 849e36f8653d4d1f9a7fb1df8fb913a9 Jun 10 13:20:15.984285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesSnapshotTestJSON-1698707508-project Jun 10 13:20:15.984285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6679e3fc-124e-4196-a5e6-0825363a9ac8) Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'server_id': 'a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744', 'id': '802588c0-10d5-4ca0-8cbb-585dee61c94e'} Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jun 10 13:20:15.985461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6679e3fc-124e-4196-a5e6-0825363a9ac8) Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:20:15.987849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6679e3fc-124e-4196-a5e6-0825363a9ac8) Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:20:15.990611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6679e3fc-124e-4196-a5e6-0825363a9ac8) Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:20:15.993055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6679e3fc-124e-4196-a5e6-0825363a9ac8) Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6679e3fc-124e-4196-a5e6-0825363a9ac8) Jun 10 13:20:15.996574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:20:15.999427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-03541bab-8ff7-4826-9641-b3350ed1c65f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6679e3fc-124e-4196-a5e6-0825363a9ac8) {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:20:15.999427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-03541bab-8ff7-4826-9641-b3350ed1c65f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e" status: 400 len: 269 microversion: 2.1 time: 60.693899 Jun 10 13:20:15.999427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 738/1497] 10.222.0.25 () {58 vars in 1415 bytes} [Fri Jun 10 13:19:15 2022] DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e => generated 269 bytes in 60695 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jun 10 13:20:15.999427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2662a129-b6b1-46fa-80e5-c5877f712bcb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:16.000188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2662a129-b6b1-46fa-80e5-c5877f712bcb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:16.000188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2662a129-b6b1-46fa-80e5-c5877f712bcb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:16.000188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2662a129-b6b1-46fa-80e5-c5877f712bcb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:16.000188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2662a129-b6b1-46fa-80e5-c5877f712bcb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:16.000561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2662a129-b6b1-46fa-80e5-c5877f712bcb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:16.058870 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-778e4a69-c5c2-4b86-9471-b34002410e07 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:16.067534 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-778e4a69-c5c2-4b86-9471-b34002410e07 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.161242 Jun 10 13:20:16.068112 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 760/1498] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:15 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:16.072993 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3ca516c7-e8dc-45f7-a254-edb015bc43b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:16.073679 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3ca516c7-e8dc-45f7-a254-edb015bc43b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:16.079857 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ca516c7-e8dc-45f7-a254-edb015bc43b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:16.080508 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ca516c7-e8dc-45f7-a254-edb015bc43b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:16.082122 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ca516c7-e8dc-45f7-a254-edb015bc43b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:16.082957 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ca516c7-e8dc-45f7-a254-edb015bc43b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:16.145272 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2662a129-b6b1-46fa-80e5-c5877f712bcb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:16.153724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2662a129-b6b1-46fa-80e5-c5877f712bcb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.167057 Jun 10 13:20:16.154383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 739/1499] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:15 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:16.159280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-916a7747-98eb-4340-83fb-f807bf1c46d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:16.159991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-916a7747-98eb-4340-83fb-f807bf1c46d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:16.165827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-916a7747-98eb-4340-83fb-f807bf1c46d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:16.166353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-916a7747-98eb-4340-83fb-f807bf1c46d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:16.167309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-916a7747-98eb-4340-83fb-f807bf1c46d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:16.167866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-916a7747-98eb-4340-83fb-f807bf1c46d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:16.298618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-916a7747-98eb-4340-83fb-f807bf1c46d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:16.306873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-916a7747-98eb-4340-83fb-f807bf1c46d4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.151306 Jun 10 13:20:16.307429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 740/1500] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:16 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:16.311900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d720161c-5190-4a2f-89f5-40201c17c358 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:16.312492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d720161c-5190-4a2f-89f5-40201c17c358 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:16.318761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d720161c-5190-4a2f-89f5-40201c17c358 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:16.319209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d720161c-5190-4a2f-89f5-40201c17c358 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:16.319964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d720161c-5190-4a2f-89f5-40201c17c358 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:16.320391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d720161c-5190-4a2f-89f5-40201c17c358 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:16.452631 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d720161c-5190-4a2f-89f5-40201c17c358 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:16.461167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d720161c-5190-4a2f-89f5-40201c17c358 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.152734 Jun 10 13:20:16.461744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 741/1501] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:16 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:17.245391 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:20:17.245956 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Fri Jun 10 13:20:17 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e (ip 10.222.0.25) !!! Jun 10 13:20:17.324525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4440d056-38d3-4dc1-ac91-c9e74f1814eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:17.325159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4440d056-38d3-4dc1-ac91-c9e74f1814eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:17.331736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4440d056-38d3-4dc1-ac91-c9e74f1814eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:17.332179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4440d056-38d3-4dc1-ac91-c9e74f1814eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:17.332974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4440d056-38d3-4dc1-ac91-c9e74f1814eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:17.333409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4440d056-38d3-4dc1-ac91-c9e74f1814eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:17.458743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4440d056-38d3-4dc1-ac91-c9e74f1814eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:17.466122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4440d056-38d3-4dc1-ac91-c9e74f1814eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145666 Jun 10 13:20:17.466678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 742/1502] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:17 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:17.478054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-146d480e-028d-478e-b727-2f2a52330207 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:17.478677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-146d480e-028d-478e-b727-2f2a52330207 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:17.484571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-146d480e-028d-478e-b727-2f2a52330207 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:17.485076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-146d480e-028d-478e-b727-2f2a52330207 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:17.485831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-146d480e-028d-478e-b727-2f2a52330207 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:17.486262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-146d480e-028d-478e-b727-2f2a52330207 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:17.614357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-146d480e-028d-478e-b727-2f2a52330207 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:17.622288 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-146d480e-028d-478e-b727-2f2a52330207 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147766 Jun 10 13:20:17.622853 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 743/1503] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:17 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:18.483812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-52792d3d-417b-4133-8064-4d895e50cedc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:18.484444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-52792d3d-417b-4133-8064-4d895e50cedc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:18.491243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52792d3d-417b-4133-8064-4d895e50cedc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:18.491702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52792d3d-417b-4133-8064-4d895e50cedc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:18.492454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52792d3d-417b-4133-8064-4d895e50cedc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:18.492903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-52792d3d-417b-4133-8064-4d895e50cedc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:18.620183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-52792d3d-417b-4133-8064-4d895e50cedc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:18.627501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-52792d3d-417b-4133-8064-4d895e50cedc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.147589 Jun 10 13:20:18.628682 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 744/1504] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:18 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:18.638871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-650e4724-e1b7-4eb2-96ea-df35f2974fb3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:18.639508 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-650e4724-e1b7-4eb2-96ea-df35f2974fb3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:18.645123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-650e4724-e1b7-4eb2-96ea-df35f2974fb3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:18.645561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-650e4724-e1b7-4eb2-96ea-df35f2974fb3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:18.646304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-650e4724-e1b7-4eb2-96ea-df35f2974fb3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:18.646730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-650e4724-e1b7-4eb2-96ea-df35f2974fb3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:18.775764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-650e4724-e1b7-4eb2-96ea-df35f2974fb3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:18.783643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-650e4724-e1b7-4eb2-96ea-df35f2974fb3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148367 Jun 10 13:20:18.784710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 745/1505] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:18 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:19.644846 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ccbb0641-b6c8-47b9-9b73-f01132a9d192 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:19.645469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ccbb0641-b6c8-47b9-9b73-f01132a9d192 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:19.651860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ccbb0641-b6c8-47b9-9b73-f01132a9d192 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:19.652306 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ccbb0641-b6c8-47b9-9b73-f01132a9d192 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:19.653103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ccbb0641-b6c8-47b9-9b73-f01132a9d192 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:19.653535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ccbb0641-b6c8-47b9-9b73-f01132a9d192 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:19.781797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ccbb0641-b6c8-47b9-9b73-f01132a9d192 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:19.789945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ccbb0641-b6c8-47b9-9b73-f01132a9d192 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.148969 Jun 10 13:20:19.790510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 746/1506] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:19 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:19.801076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2864981f-b7b4-46f6-b7b9-0713ae9b081a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:19.801674 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2864981f-b7b4-46f6-b7b9-0713ae9b081a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:19.807457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2864981f-b7b4-46f6-b7b9-0713ae9b081a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:19.810815 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2864981f-b7b4-46f6-b7b9-0713ae9b081a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:19.810815 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2864981f-b7b4-46f6-b7b9-0713ae9b081a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:19.810815 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2864981f-b7b4-46f6-b7b9-0713ae9b081a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:19.936706 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2864981f-b7b4-46f6-b7b9-0713ae9b081a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:19.944558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2864981f-b7b4-46f6-b7b9-0713ae9b081a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147666 Jun 10 13:20:19.945155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 747/1507] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:19 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:20.564707 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:20:20.565710 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Fri Jun 10 13:20:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e (ip 10.222.0.25) !!! Jun 10 13:20:20.806725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9e647ac4-e2e4-4311-ac2c-0a01915ac13c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:20.807328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9e647ac4-e2e4-4311-ac2c-0a01915ac13c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:20.813830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e647ac4-e2e4-4311-ac2c-0a01915ac13c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:20.814273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e647ac4-e2e4-4311-ac2c-0a01915ac13c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:20.815078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e647ac4-e2e4-4311-ac2c-0a01915ac13c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:20.816127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e647ac4-e2e4-4311-ac2c-0a01915ac13c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:20.943065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9e647ac4-e2e4-4311-ac2c-0a01915ac13c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:20.950461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9e647ac4-e2e4-4311-ac2c-0a01915ac13c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.147574 Jun 10 13:20:20.951020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 748/1508] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:20 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:20.961022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a40c8179-db1c-4bb0-be3d-0ada5b86a4ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:20.961626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a40c8179-db1c-4bb0-be3d-0ada5b86a4ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:20.967244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a40c8179-db1c-4bb0-be3d-0ada5b86a4ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:20.967695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a40c8179-db1c-4bb0-be3d-0ada5b86a4ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:20.968451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a40c8179-db1c-4bb0-be3d-0ada5b86a4ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:20.968928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a40c8179-db1c-4bb0-be3d-0ada5b86a4ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:21.103334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a40c8179-db1c-4bb0-be3d-0ada5b86a4ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:21.111976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a40c8179-db1c-4bb0-be3d-0ada5b86a4ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.154462 Jun 10 13:20:21.113226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 749/1509] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:20 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:21.967318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b9066dd1-05e1-415c-8593-3a58eb660829 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:21.968034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b9066dd1-05e1-415c-8593-3a58eb660829 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:21.974403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9066dd1-05e1-415c-8593-3a58eb660829 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:21.974852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9066dd1-05e1-415c-8593-3a58eb660829 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:21.975606 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9066dd1-05e1-415c-8593-3a58eb660829 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:21.976057 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9066dd1-05e1-415c-8593-3a58eb660829 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:22.102631 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b9066dd1-05e1-415c-8593-3a58eb660829 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:22.109648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b9066dd1-05e1-415c-8593-3a58eb660829 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146147 Jun 10 13:20:22.110290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 750/1510] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:21 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:22.129246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-06f9c1b6-00c8-4678-95c1-ed9fc2483d4f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:22.129873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-06f9c1b6-00c8-4678-95c1-ed9fc2483d4f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:22.135755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06f9c1b6-00c8-4678-95c1-ed9fc2483d4f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:22.136207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06f9c1b6-00c8-4678-95c1-ed9fc2483d4f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:22.137008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06f9c1b6-00c8-4678-95c1-ed9fc2483d4f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:22.137447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06f9c1b6-00c8-4678-95c1-ed9fc2483d4f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:22.265971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-06f9c1b6-00c8-4678-95c1-ed9fc2483d4f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:22.274406 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-06f9c1b6-00c8-4678-95c1-ed9fc2483d4f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.149369 Jun 10 13:20:22.274974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 751/1511] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:22 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:23.126240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e41bbe63-80b3-4b5a-8195-b2bc82845b37 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:23.126933 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e41bbe63-80b3-4b5a-8195-b2bc82845b37 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:23.133900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e41bbe63-80b3-4b5a-8195-b2bc82845b37 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:23.134339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e41bbe63-80b3-4b5a-8195-b2bc82845b37 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:23.135082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e41bbe63-80b3-4b5a-8195-b2bc82845b37 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:23.135505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e41bbe63-80b3-4b5a-8195-b2bc82845b37 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:23.363862 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e41bbe63-80b3-4b5a-8195-b2bc82845b37 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:23.371309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e41bbe63-80b3-4b5a-8195-b2bc82845b37 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.248985 Jun 10 13:20:23.371865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 752/1512] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:23 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:23.376103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-26c07395-2938-4374-b865-9edae98dfb48 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:23.376692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-26c07395-2938-4374-b865-9edae98dfb48 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:23.382124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26c07395-2938-4374-b865-9edae98dfb48 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:23.382567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26c07395-2938-4374-b865-9edae98dfb48 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:23.383310 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26c07395-2938-4374-b865-9edae98dfb48 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:23.383739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26c07395-2938-4374-b865-9edae98dfb48 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:23.508448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-26c07395-2938-4374-b865-9edae98dfb48 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:23.516250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-26c07395-2938-4374-b865-9edae98dfb48 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143405 Jun 10 13:20:23.516744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 753/1513] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:23 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:24.387603 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3dcae2b7-05d4-4bdb-a694-0e23d03b22d2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:24.388277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3dcae2b7-05d4-4bdb-a694-0e23d03b22d2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:24.394831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3dcae2b7-05d4-4bdb-a694-0e23d03b22d2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:24.395274 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3dcae2b7-05d4-4bdb-a694-0e23d03b22d2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:24.396056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3dcae2b7-05d4-4bdb-a694-0e23d03b22d2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:24.396484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3dcae2b7-05d4-4bdb-a694-0e23d03b22d2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:24.522966 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3dcae2b7-05d4-4bdb-a694-0e23d03b22d2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:24.530312 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3dcae2b7-05d4-4bdb-a694-0e23d03b22d2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146555 Jun 10 13:20:24.530872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 754/1514] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:24 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:24.535766 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2dcad545-5156-425c-a239-d537009b742d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:24.536351 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2dcad545-5156-425c-a239-d537009b742d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:24.541910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2dcad545-5156-425c-a239-d537009b742d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:24.542355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2dcad545-5156-425c-a239-d537009b742d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:24.543101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2dcad545-5156-425c-a239-d537009b742d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:24.543525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2dcad545-5156-425c-a239-d537009b742d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:24.560291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:20:24.666505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2dcad545-5156-425c-a239-d537009b742d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:24.675337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2dcad545-5156-425c-a239-d537009b742d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143439 Jun 10 13:20:24.675847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 755/1515] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:24 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:25.548203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5d5c71c3-a26f-49f1-b8a9-c7d81fc446c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:25.548830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5d5c71c3-a26f-49f1-b8a9-c7d81fc446c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:25.555998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d5c71c3-a26f-49f1-b8a9-c7d81fc446c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:25.556465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d5c71c3-a26f-49f1-b8a9-c7d81fc446c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:25.557245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d5c71c3-a26f-49f1-b8a9-c7d81fc446c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:25.557673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d5c71c3-a26f-49f1-b8a9-c7d81fc446c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:25.686820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5d5c71c3-a26f-49f1-b8a9-c7d81fc446c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:25.694209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5d5c71c3-a26f-49f1-b8a9-c7d81fc446c6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149755 Jun 10 13:20:25.694772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 756/1516] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:25 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:25.699096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1f29778d-5ce3-46dd-8e7c-6181babdc22c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:25.699679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1f29778d-5ce3-46dd-8e7c-6181babdc22c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:25.705576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1f29778d-5ce3-46dd-8e7c-6181babdc22c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:25.706017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1f29778d-5ce3-46dd-8e7c-6181babdc22c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:25.706773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1f29778d-5ce3-46dd-8e7c-6181babdc22c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:25.707198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1f29778d-5ce3-46dd-8e7c-6181babdc22c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:25.830285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1f29778d-5ce3-46dd-8e7c-6181babdc22c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:25.838682 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1f29778d-5ce3-46dd-8e7c-6181babdc22c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.142996 Jun 10 13:20:25.839313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 757/1517] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:25 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:26.079814 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=4 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:20:26.101975 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=10 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:20:26.714707 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7f6f5a70-4b13-4b9c-ad72-7620deaf8bae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:26.715317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7f6f5a70-4b13-4b9c-ad72-7620deaf8bae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:26.722556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f6f5a70-4b13-4b9c-ad72-7620deaf8bae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:26.722998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f6f5a70-4b13-4b9c-ad72-7620deaf8bae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:26.723755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f6f5a70-4b13-4b9c-ad72-7620deaf8bae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:26.724185 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f6f5a70-4b13-4b9c-ad72-7620deaf8bae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:26.855551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7f6f5a70-4b13-4b9c-ad72-7620deaf8bae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:26.859867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7f6f5a70-4b13-4b9c-ad72-7620deaf8bae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149219 Jun 10 13:20:26.860431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 758/1518] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:26 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:26.864725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-72872db6-5808-4fba-b74e-c07d3056b4f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:26.865354 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-72872db6-5808-4fba-b74e-c07d3056b4f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:26.870835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72872db6-5808-4fba-b74e-c07d3056b4f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:26.871279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72872db6-5808-4fba-b74e-c07d3056b4f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:26.872048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72872db6-5808-4fba-b74e-c07d3056b4f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:26.872464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-72872db6-5808-4fba-b74e-c07d3056b4f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:26.995709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-72872db6-5808-4fba-b74e-c07d3056b4f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:27.004282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-72872db6-5808-4fba-b74e-c07d3056b4f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.142839 Jun 10 13:20:27.004867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 759/1519] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:26 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:27.877159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-693e2ffd-a911-422c-ab8a-3409cef9030a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:27.878447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-693e2ffd-a911-422c-ab8a-3409cef9030a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:27.884633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-693e2ffd-a911-422c-ab8a-3409cef9030a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:27.885111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-693e2ffd-a911-422c-ab8a-3409cef9030a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:27.885870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-693e2ffd-a911-422c-ab8a-3409cef9030a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:27.886302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-693e2ffd-a911-422c-ab8a-3409cef9030a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:28.014824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-693e2ffd-a911-422c-ab8a-3409cef9030a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:28.022866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-693e2ffd-a911-422c-ab8a-3409cef9030a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149942 Jun 10 13:20:28.023429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 760/1520] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:27 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:28.027721 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c72da33d-5516-4449-84bb-d5d66dd45ad2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:28.028315 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c72da33d-5516-4449-84bb-d5d66dd45ad2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:28.033794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c72da33d-5516-4449-84bb-d5d66dd45ad2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:28.034238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c72da33d-5516-4449-84bb-d5d66dd45ad2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:28.035021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c72da33d-5516-4449-84bb-d5d66dd45ad2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:28.035466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c72da33d-5516-4449-84bb-d5d66dd45ad2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:28.161694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c72da33d-5516-4449-84bb-d5d66dd45ad2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:28.169484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c72da33d-5516-4449-84bb-d5d66dd45ad2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145121 Jun 10 13:20:28.170044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 761/1521] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:28 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:29.039486 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-59b549b9-4771-4afd-9d24-d6fdd5f575c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:29.040149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-59b549b9-4771-4afd-9d24-d6fdd5f575c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:29.046706 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-59b549b9-4771-4afd-9d24-d6fdd5f575c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:29.047154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-59b549b9-4771-4afd-9d24-d6fdd5f575c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:29.047913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-59b549b9-4771-4afd-9d24-d6fdd5f575c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:29.048345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-59b549b9-4771-4afd-9d24-d6fdd5f575c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:29.172834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-59b549b9-4771-4afd-9d24-d6fdd5f575c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:29.180651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-59b549b9-4771-4afd-9d24-d6fdd5f575c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145043 Jun 10 13:20:29.181241 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 762/1522] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:29 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:29.187410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-95abf5af-5952-4035-a407-5a1e6d10d5f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:29.188015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-95abf5af-5952-4035-a407-5a1e6d10d5f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:29.194412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95abf5af-5952-4035-a407-5a1e6d10d5f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:29.194861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95abf5af-5952-4035-a407-5a1e6d10d5f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:29.195620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95abf5af-5952-4035-a407-5a1e6d10d5f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:29.196054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95abf5af-5952-4035-a407-5a1e6d10d5f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:29.322701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-95abf5af-5952-4035-a407-5a1e6d10d5f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:29.330738 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-95abf5af-5952-4035-a407-5a1e6d10d5f7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146849 Jun 10 13:20:29.331302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 763/1523] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:29 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:30.198070 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-86cb263f-b16a-49b8-9ffa-c38bebbfee32 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:30.198737 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-86cb263f-b16a-49b8-9ffa-c38bebbfee32 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:30.205319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-86cb263f-b16a-49b8-9ffa-c38bebbfee32 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:30.205760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-86cb263f-b16a-49b8-9ffa-c38bebbfee32 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:30.206503 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-86cb263f-b16a-49b8-9ffa-c38bebbfee32 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:30.206932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-86cb263f-b16a-49b8-9ffa-c38bebbfee32 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:30.333358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-86cb263f-b16a-49b8-9ffa-c38bebbfee32 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:30.340862 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-86cb263f-b16a-49b8-9ffa-c38bebbfee32 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146567 Jun 10 13:20:30.341660 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 764/1524] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:30 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:30.347448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d05139b0-93f0-45b7-82fc-0cce4c3c329c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:30.348049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d05139b0-93f0-45b7-82fc-0cce4c3c329c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:30.354012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d05139b0-93f0-45b7-82fc-0cce4c3c329c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:30.354452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d05139b0-93f0-45b7-82fc-0cce4c3c329c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:30.355209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d05139b0-93f0-45b7-82fc-0cce4c3c329c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:30.355643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d05139b0-93f0-45b7-82fc-0cce4c3c329c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:30.486730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d05139b0-93f0-45b7-82fc-0cce4c3c329c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:30.494483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d05139b0-93f0-45b7-82fc-0cce4c3c329c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.151140 Jun 10 13:20:30.495045 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 765/1525] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:30 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:31.358484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7f5deaf7-ee24-4879-860b-a190c44c1e48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:31.359077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7f5deaf7-ee24-4879-860b-a190c44c1e48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:31.365816 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f5deaf7-ee24-4879-860b-a190c44c1e48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:31.366259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f5deaf7-ee24-4879-860b-a190c44c1e48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:31.367045 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f5deaf7-ee24-4879-860b-a190c44c1e48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:31.367481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f5deaf7-ee24-4879-860b-a190c44c1e48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:31.499920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7f5deaf7-ee24-4879-860b-a190c44c1e48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:31.508111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7f5deaf7-ee24-4879-860b-a190c44c1e48 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.153487 Jun 10 13:20:31.508681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 766/1526] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:31 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:31.513129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-eec2b450-e127-4cf7-8512-7aa4f3ebaa92 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:31.513726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-eec2b450-e127-4cf7-8512-7aa4f3ebaa92 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:31.519252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eec2b450-e127-4cf7-8512-7aa4f3ebaa92 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:31.519698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eec2b450-e127-4cf7-8512-7aa4f3ebaa92 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:31.520452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eec2b450-e127-4cf7-8512-7aa4f3ebaa92 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:31.520921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eec2b450-e127-4cf7-8512-7aa4f3ebaa92 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:31.645430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-eec2b450-e127-4cf7-8512-7aa4f3ebaa92 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:31.653621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-eec2b450-e127-4cf7-8512-7aa4f3ebaa92 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143923 Jun 10 13:20:31.654206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 767/1527] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:31 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:32.525109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-74e06cc2-d883-49ff-bb96-9e240849e9e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:32.525740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-74e06cc2-d883-49ff-bb96-9e240849e9e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:32.532041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74e06cc2-d883-49ff-bb96-9e240849e9e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:32.532484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74e06cc2-d883-49ff-bb96-9e240849e9e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:32.533275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74e06cc2-d883-49ff-bb96-9e240849e9e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:32.533733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74e06cc2-d883-49ff-bb96-9e240849e9e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:32.660420 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-74e06cc2-d883-49ff-bb96-9e240849e9e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:32.667877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-74e06cc2-d883-49ff-bb96-9e240849e9e3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146834 Jun 10 13:20:32.668447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 768/1528] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:32 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:32.672789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3f8c06ec-7a4c-4fcf-828c-d7f3794bac00 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:32.673384 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3f8c06ec-7a4c-4fcf-828c-d7f3794bac00 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:32.679441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f8c06ec-7a4c-4fcf-828c-d7f3794bac00 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:32.679886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f8c06ec-7a4c-4fcf-828c-d7f3794bac00 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:32.680645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f8c06ec-7a4c-4fcf-828c-d7f3794bac00 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:32.681103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f8c06ec-7a4c-4fcf-828c-d7f3794bac00 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:32.805185 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f8c06ec-7a4c-4fcf-828c-d7f3794bac00 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:32.813077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3f8c06ec-7a4c-4fcf-828c-d7f3794bac00 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143624 Jun 10 13:20:32.813645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 769/1529] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:32 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:33.689586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a5b96d4b-9869-4daa-ada3-b1a1cb729e8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:33.689586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a5b96d4b-9869-4daa-ada3-b1a1cb729e8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:33.692471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a5b96d4b-9869-4daa-ada3-b1a1cb729e8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:33.692945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a5b96d4b-9869-4daa-ada3-b1a1cb729e8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:33.693702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a5b96d4b-9869-4daa-ada3-b1a1cb729e8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:33.694132 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a5b96d4b-9869-4daa-ada3-b1a1cb729e8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:33.822561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a5b96d4b-9869-4daa-ada3-b1a1cb729e8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:33.831018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a5b96d4b-9869-4daa-ada3-b1a1cb729e8b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.148805 Jun 10 13:20:33.831577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 770/1530] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:33 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:33.835873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-63677179-a187-4b4b-9ef7-cccee602ccbd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:33.836728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-63677179-a187-4b4b-9ef7-cccee602ccbd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:33.842252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63677179-a187-4b4b-9ef7-cccee602ccbd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:33.842696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63677179-a187-4b4b-9ef7-cccee602ccbd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:33.843433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63677179-a187-4b4b-9ef7-cccee602ccbd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:33.843892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63677179-a187-4b4b-9ef7-cccee602ccbd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:33.967949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-63677179-a187-4b4b-9ef7-cccee602ccbd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:33.976135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-63677179-a187-4b4b-9ef7-cccee602ccbd tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143604 Jun 10 13:20:33.976711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 771/1531] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:33 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:34.847778 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ef93dd77-9dc7-4246-9240-3f52da3cc373 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:34.848440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ef93dd77-9dc7-4246-9240-3f52da3cc373 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:34.855070 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ef93dd77-9dc7-4246-9240-3f52da3cc373 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:34.855504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ef93dd77-9dc7-4246-9240-3f52da3cc373 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:34.856248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ef93dd77-9dc7-4246-9240-3f52da3cc373 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:34.856672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ef93dd77-9dc7-4246-9240-3f52da3cc373 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:34.860581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:20:34.982547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ef93dd77-9dc7-4246-9240-3f52da3cc373 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:34.989950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ef93dd77-9dc7-4246-9240-3f52da3cc373 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146011 Jun 10 13:20:34.990511 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 772/1532] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:34 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:34.995114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b41a3920-9bca-40af-9969-8a04b0b3b869 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:34.995707 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b41a3920-9bca-40af-9969-8a04b0b3b869 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:35.002922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b41a3920-9bca-40af-9969-8a04b0b3b869 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:35.003367 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b41a3920-9bca-40af-9969-8a04b0b3b869 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:35.004119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b41a3920-9bca-40af-9969-8a04b0b3b869 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:35.004548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b41a3920-9bca-40af-9969-8a04b0b3b869 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:35.128317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b41a3920-9bca-40af-9969-8a04b0b3b869 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:35.136078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b41a3920-9bca-40af-9969-8a04b0b3b869 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144645 Jun 10 13:20:35.136635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 773/1533] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:34 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:36.008356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-be8be5cf-83d6-43a0-b814-d6b09fd8dacf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:36.009065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-be8be5cf-83d6-43a0-b814-d6b09fd8dacf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:36.016731 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be8be5cf-83d6-43a0-b814-d6b09fd8dacf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:36.017306 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be8be5cf-83d6-43a0-b814-d6b09fd8dacf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:36.018086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be8be5cf-83d6-43a0-b814-d6b09fd8dacf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:36.018513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be8be5cf-83d6-43a0-b814-d6b09fd8dacf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:36.144393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-be8be5cf-83d6-43a0-b814-d6b09fd8dacf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:36.152142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-be8be5cf-83d6-43a0-b814-d6b09fd8dacf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.148385 Jun 10 13:20:36.152707 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 774/1534] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:36 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:36.157035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1257eb3f-76cb-4e8a-b931-2d642342fecc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:36.157634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1257eb3f-76cb-4e8a-b931-2d642342fecc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:36.163605 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1257eb3f-76cb-4e8a-b931-2d642342fecc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:36.164049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1257eb3f-76cb-4e8a-b931-2d642342fecc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:36.164830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1257eb3f-76cb-4e8a-b931-2d642342fecc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:36.165266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1257eb3f-76cb-4e8a-b931-2d642342fecc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:36.290932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1257eb3f-76cb-4e8a-b931-2d642342fecc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:36.298652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1257eb3f-76cb-4e8a-b931-2d642342fecc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144992 Jun 10 13:20:36.299838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 775/1535] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:36 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:37.168999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8e88a25b-92f9-4b40-ad66-6e8fee1eaae8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:37.169688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8e88a25b-92f9-4b40-ad66-6e8fee1eaae8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:37.176902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8e88a25b-92f9-4b40-ad66-6e8fee1eaae8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:37.177348 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8e88a25b-92f9-4b40-ad66-6e8fee1eaae8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:37.178114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8e88a25b-92f9-4b40-ad66-6e8fee1eaae8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:37.178544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8e88a25b-92f9-4b40-ad66-6e8fee1eaae8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:37.307703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8e88a25b-92f9-4b40-ad66-6e8fee1eaae8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:37.315961 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8e88a25b-92f9-4b40-ad66-6e8fee1eaae8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.150866 Jun 10 13:20:37.325507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 776/1536] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:37 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:37.325507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6783228e-76e4-4d96-880e-267958b2ac71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:37.325507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6783228e-76e4-4d96-880e-267958b2ac71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:37.327405 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6783228e-76e4-4d96-880e-267958b2ac71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:37.327851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6783228e-76e4-4d96-880e-267958b2ac71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:37.328608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6783228e-76e4-4d96-880e-267958b2ac71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:37.329071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6783228e-76e4-4d96-880e-267958b2ac71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:37.462118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6783228e-76e4-4d96-880e-267958b2ac71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:37.470033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6783228e-76e4-4d96-880e-267958b2ac71 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.152505 Jun 10 13:20:37.470613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 777/1537] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:37 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:38.333592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9337b63a-6685-4447-8fd7-8f59db28a4c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:38.334248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9337b63a-6685-4447-8fd7-8f59db28a4c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:38.340730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9337b63a-6685-4447-8fd7-8f59db28a4c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:38.341210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9337b63a-6685-4447-8fd7-8f59db28a4c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:38.341968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9337b63a-6685-4447-8fd7-8f59db28a4c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:38.342397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9337b63a-6685-4447-8fd7-8f59db28a4c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:38.467852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9337b63a-6685-4447-8fd7-8f59db28a4c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:38.475255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9337b63a-6685-4447-8fd7-8f59db28a4c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146248 Jun 10 13:20:38.475826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 778/1538] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:38 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:38.487026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c887fd54-7f61-4305-aeea-e36d2e4cb559 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:38.487623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c887fd54-7f61-4305-aeea-e36d2e4cb559 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:38.493286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c887fd54-7f61-4305-aeea-e36d2e4cb559 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:38.493729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c887fd54-7f61-4305-aeea-e36d2e4cb559 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:38.494475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c887fd54-7f61-4305-aeea-e36d2e4cb559 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:38.494900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c887fd54-7f61-4305-aeea-e36d2e4cb559 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:38.617821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c887fd54-7f61-4305-aeea-e36d2e4cb559 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:38.625662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c887fd54-7f61-4305-aeea-e36d2e4cb559 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.142121 Jun 10 13:20:38.626227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 779/1539] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:38 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:39.492292 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4ba90205-ce2f-40ec-8f18-27fbfb91c429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:39.493167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4ba90205-ce2f-40ec-8f18-27fbfb91c429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:39.499781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4ba90205-ce2f-40ec-8f18-27fbfb91c429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:39.500223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4ba90205-ce2f-40ec-8f18-27fbfb91c429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:39.501002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4ba90205-ce2f-40ec-8f18-27fbfb91c429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:39.501428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4ba90205-ce2f-40ec-8f18-27fbfb91c429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:39.629125 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4ba90205-ce2f-40ec-8f18-27fbfb91c429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:39.639633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4ba90205-ce2f-40ec-8f18-27fbfb91c429 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.151073 Jun 10 13:20:39.640209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 780/1540] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:39 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:39.645236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9ef7f2dc-993c-456e-9cd2-669118f8df29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:39.645805 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9ef7f2dc-993c-456e-9cd2-669118f8df29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:39.651915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ef7f2dc-993c-456e-9cd2-669118f8df29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:39.652358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ef7f2dc-993c-456e-9cd2-669118f8df29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:39.653163 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ef7f2dc-993c-456e-9cd2-669118f8df29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:39.653593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ef7f2dc-993c-456e-9cd2-669118f8df29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:39.778285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9ef7f2dc-993c-456e-9cd2-669118f8df29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:39.786663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9ef7f2dc-993c-456e-9cd2-669118f8df29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145527 Jun 10 13:20:39.787222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 781/1541] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:39 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:40.658878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-535a7a50-de79-4eb2-ab50-0ac90a086690 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:40.659506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-535a7a50-de79-4eb2-ab50-0ac90a086690 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:40.665822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-535a7a50-de79-4eb2-ab50-0ac90a086690 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:40.666263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-535a7a50-de79-4eb2-ab50-0ac90a086690 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:40.667009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-535a7a50-de79-4eb2-ab50-0ac90a086690 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:40.667437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-535a7a50-de79-4eb2-ab50-0ac90a086690 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:40.798171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-535a7a50-de79-4eb2-ab50-0ac90a086690 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:40.804015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-535a7a50-de79-4eb2-ab50-0ac90a086690 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149565 Jun 10 13:20:40.804571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 782/1542] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:40 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:40.809157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-30910b00-6a73-4532-9497-fc5ae0cca8a9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:40.809749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-30910b00-6a73-4532-9497-fc5ae0cca8a9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:40.815420 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-30910b00-6a73-4532-9497-fc5ae0cca8a9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:40.815861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-30910b00-6a73-4532-9497-fc5ae0cca8a9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:40.816609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-30910b00-6a73-4532-9497-fc5ae0cca8a9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:40.817122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-30910b00-6a73-4532-9497-fc5ae0cca8a9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:40.940019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-30910b00-6a73-4532-9497-fc5ae0cca8a9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:40.947849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-30910b00-6a73-4532-9497-fc5ae0cca8a9 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.142320 Jun 10 13:20:40.948596 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 783/1543] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:40 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:41.821317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d76c2300-4f95-41e8-909b-b551e56dbb64 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:41.821981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d76c2300-4f95-41e8-909b-b551e56dbb64 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:41.828427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d76c2300-4f95-41e8-909b-b551e56dbb64 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:41.828918 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d76c2300-4f95-41e8-909b-b551e56dbb64 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:41.829679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d76c2300-4f95-41e8-909b-b551e56dbb64 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:41.830129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d76c2300-4f95-41e8-909b-b551e56dbb64 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:41.954693 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d76c2300-4f95-41e8-909b-b551e56dbb64 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:41.962530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d76c2300-4f95-41e8-909b-b551e56dbb64 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145070 Jun 10 13:20:41.963092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 784/1544] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:41 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:41.967343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fd986900-3ff7-403f-bfa3-e434e3356d40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:41.967941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fd986900-3ff7-403f-bfa3-e434e3356d40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:41.973476 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fd986900-3ff7-403f-bfa3-e434e3356d40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:41.973914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fd986900-3ff7-403f-bfa3-e434e3356d40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:41.974657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fd986900-3ff7-403f-bfa3-e434e3356d40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:41.975082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fd986900-3ff7-403f-bfa3-e434e3356d40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:42.104323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fd986900-3ff7-403f-bfa3-e434e3356d40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:42.113153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fd986900-3ff7-403f-bfa3-e434e3356d40 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.149099 Jun 10 13:20:42.113724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 785/1545] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:41 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:42.979592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-84fcad64-d856-48f6-ba6d-69b548682cd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:42.980283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-84fcad64-d856-48f6-ba6d-69b548682cd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:42.986806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84fcad64-d856-48f6-ba6d-69b548682cd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:42.987252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84fcad64-d856-48f6-ba6d-69b548682cd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:42.988015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84fcad64-d856-48f6-ba6d-69b548682cd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:42.988482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84fcad64-d856-48f6-ba6d-69b548682cd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:43.116389 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-84fcad64-d856-48f6-ba6d-69b548682cd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:43.123763 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-84fcad64-d856-48f6-ba6d-69b548682cd5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.148314 Jun 10 13:20:43.124331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 786/1546] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:42 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:43.130016 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0f087f1b-dd35-45ef-85f6-51519fc2b13a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:43.131604 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0f087f1b-dd35-45ef-85f6-51519fc2b13a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:43.136668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f087f1b-dd35-45ef-85f6-51519fc2b13a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:43.137186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f087f1b-dd35-45ef-85f6-51519fc2b13a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:43.137945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f087f1b-dd35-45ef-85f6-51519fc2b13a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:43.138380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f087f1b-dd35-45ef-85f6-51519fc2b13a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:43.265011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0f087f1b-dd35-45ef-85f6-51519fc2b13a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:43.272873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0f087f1b-dd35-45ef-85f6-51519fc2b13a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146995 Jun 10 13:20:43.273431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 787/1547] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:43 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:44.141643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f8c98482-9d13-47fb-8e69-9179ca4ff265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:44.142800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f8c98482-9d13-47fb-8e69-9179ca4ff265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:44.146156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:20:44.149539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8c98482-9d13-47fb-8e69-9179ca4ff265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:44.154908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8c98482-9d13-47fb-8e69-9179ca4ff265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:44.154908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8c98482-9d13-47fb-8e69-9179ca4ff265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:44.154908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8c98482-9d13-47fb-8e69-9179ca4ff265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:44.275407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f8c98482-9d13-47fb-8e69-9179ca4ff265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:44.278851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:20:44.283817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f8c98482-9d13-47fb-8e69-9179ca4ff265 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145384 Jun 10 13:20:44.284420 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 788/1548] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:44 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:44.289365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-644475ff-58da-4256-b862-35253164eb6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:44.292174 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-644475ff-58da-4256-b862-35253164eb6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:44.295581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-644475ff-58da-4256-b862-35253164eb6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:44.296027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-644475ff-58da-4256-b862-35253164eb6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:44.296805 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-644475ff-58da-4256-b862-35253164eb6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:44.297249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-644475ff-58da-4256-b862-35253164eb6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:44.423993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-644475ff-58da-4256-b862-35253164eb6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:44.431950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-644475ff-58da-4256-b862-35253164eb6e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146024 Jun 10 13:20:44.432523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 789/1549] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:44 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:45.301449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9360dcfa-6c16-4435-9d10-0be7b74df79c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:45.302134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9360dcfa-6c16-4435-9d10-0be7b74df79c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:45.308486 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9360dcfa-6c16-4435-9d10-0be7b74df79c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:45.308990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9360dcfa-6c16-4435-9d10-0be7b74df79c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:45.309744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9360dcfa-6c16-4435-9d10-0be7b74df79c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:45.310177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9360dcfa-6c16-4435-9d10-0be7b74df79c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:45.436436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9360dcfa-6c16-4435-9d10-0be7b74df79c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:45.443952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9360dcfa-6c16-4435-9d10-0be7b74df79c tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146997 Jun 10 13:20:45.444535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 790/1550] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:45 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:45.448822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cb91a3ec-d0fa-4423-bb5d-54397f50ee3a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:45.449424 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cb91a3ec-d0fa-4423-bb5d-54397f50ee3a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:45.455431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb91a3ec-d0fa-4423-bb5d-54397f50ee3a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:45.455926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb91a3ec-d0fa-4423-bb5d-54397f50ee3a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:45.456697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb91a3ec-d0fa-4423-bb5d-54397f50ee3a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:45.457154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb91a3ec-d0fa-4423-bb5d-54397f50ee3a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:45.578363 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cb91a3ec-d0fa-4423-bb5d-54397f50ee3a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:45.586678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cb91a3ec-d0fa-4423-bb5d-54397f50ee3a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.141198 Jun 10 13:20:45.587247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 791/1551] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:45 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:46.461116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-06b0e072-5938-49ec-a97c-0db984d0fe60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:46.461783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-06b0e072-5938-49ec-a97c-0db984d0fe60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:46.468322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06b0e072-5938-49ec-a97c-0db984d0fe60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:46.468841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06b0e072-5938-49ec-a97c-0db984d0fe60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:46.469600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06b0e072-5938-49ec-a97c-0db984d0fe60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:46.470027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06b0e072-5938-49ec-a97c-0db984d0fe60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:46.603722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-06b0e072-5938-49ec-a97c-0db984d0fe60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:46.611683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-06b0e072-5938-49ec-a97c-0db984d0fe60 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.153828 Jun 10 13:20:46.612251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 792/1552] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:46 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:46.616483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-aa2df95f-61d9-4616-8e0c-94cafcb8d17a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:46.617107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-aa2df95f-61d9-4616-8e0c-94cafcb8d17a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:46.622639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aa2df95f-61d9-4616-8e0c-94cafcb8d17a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:46.623083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aa2df95f-61d9-4616-8e0c-94cafcb8d17a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:46.623832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aa2df95f-61d9-4616-8e0c-94cafcb8d17a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:46.624256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aa2df95f-61d9-4616-8e0c-94cafcb8d17a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:46.755767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-aa2df95f-61d9-4616-8e0c-94cafcb8d17a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:46.763838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-aa2df95f-61d9-4616-8e0c-94cafcb8d17a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.150523 Jun 10 13:20:46.764477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 793/1553] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:46 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:47.628544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b433818f-00f3-43de-b085-f5322f5dfef7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:47.629241 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b433818f-00f3-43de-b085-f5322f5dfef7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:47.635749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b433818f-00f3-43de-b085-f5322f5dfef7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:47.636195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b433818f-00f3-43de-b085-f5322f5dfef7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:47.636998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b433818f-00f3-43de-b085-f5322f5dfef7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:47.637439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b433818f-00f3-43de-b085-f5322f5dfef7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:47.764473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b433818f-00f3-43de-b085-f5322f5dfef7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:47.772433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b433818f-00f3-43de-b085-f5322f5dfef7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.147785 Jun 10 13:20:47.773036 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 794/1554] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:47 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:47.780404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e15675fc-da95-4a4b-9bde-99aa79f930e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:47.781035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e15675fc-da95-4a4b-9bde-99aa79f930e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:47.787391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e15675fc-da95-4a4b-9bde-99aa79f930e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:47.787840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e15675fc-da95-4a4b-9bde-99aa79f930e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:47.788592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e15675fc-da95-4a4b-9bde-99aa79f930e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:47.789052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e15675fc-da95-4a4b-9bde-99aa79f930e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:47.912235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e15675fc-da95-4a4b-9bde-99aa79f930e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:47.920030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e15675fc-da95-4a4b-9bde-99aa79f930e5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143159 Jun 10 13:20:47.920591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 795/1555] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:47 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:48.789529 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ebbc1a6b-10af-456e-b65f-5d0a937e78e9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:48.790182 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ebbc1a6b-10af-456e-b65f-5d0a937e78e9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:48.796480 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebbc1a6b-10af-456e-b65f-5d0a937e78e9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:48.796970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebbc1a6b-10af-456e-b65f-5d0a937e78e9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:48.797718 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebbc1a6b-10af-456e-b65f-5d0a937e78e9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:48.798142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebbc1a6b-10af-456e-b65f-5d0a937e78e9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:48.926535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ebbc1a6b-10af-456e-b65f-5d0a937e78e9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:48.934295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ebbc1a6b-10af-456e-b65f-5d0a937e78e9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.148684 Jun 10 13:20:48.934855 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 796/1556] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:48 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:48.939262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-07f42c69-f2f7-43fd-bb44-3ef364656adc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:48.939851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-07f42c69-f2f7-43fd-bb44-3ef364656adc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:48.945393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07f42c69-f2f7-43fd-bb44-3ef364656adc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:48.945842 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07f42c69-f2f7-43fd-bb44-3ef364656adc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:48.946580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07f42c69-f2f7-43fd-bb44-3ef364656adc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:48.947008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07f42c69-f2f7-43fd-bb44-3ef364656adc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:49.074212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-07f42c69-f2f7-43fd-bb44-3ef364656adc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:49.082021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-07f42c69-f2f7-43fd-bb44-3ef364656adc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146180 Jun 10 13:20:49.082588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 797/1557] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:48 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:49.951173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a07c5028-d5b2-4771-8537-52ceb1112b34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:49.951822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a07c5028-d5b2-4771-8537-52ceb1112b34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:49.958276 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a07c5028-d5b2-4771-8537-52ceb1112b34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:49.958723 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a07c5028-d5b2-4771-8537-52ceb1112b34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:49.959474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a07c5028-d5b2-4771-8537-52ceb1112b34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:49.959906 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a07c5028-d5b2-4771-8537-52ceb1112b34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:50.085723 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a07c5028-d5b2-4771-8537-52ceb1112b34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:50.093318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a07c5028-d5b2-4771-8537-52ceb1112b34 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145978 Jun 10 13:20:50.093884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 798/1558] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:49 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:50.098278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a8037260-b65c-4dc7-9199-b6007570c136 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:50.098880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a8037260-b65c-4dc7-9199-b6007570c136 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:50.104848 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8037260-b65c-4dc7-9199-b6007570c136 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:50.105303 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8037260-b65c-4dc7-9199-b6007570c136 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:50.106064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8037260-b65c-4dc7-9199-b6007570c136 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:50.106500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8037260-b65c-4dc7-9199-b6007570c136 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:50.233376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a8037260-b65c-4dc7-9199-b6007570c136 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:50.241803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a8037260-b65c-4dc7-9199-b6007570c136 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146980 Jun 10 13:20:50.242366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 799/1559] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:50 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:51.110438 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3cc4981b-3f37-4973-96e2-a77282c9e5e4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:51.111119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3cc4981b-3f37-4973-96e2-a77282c9e5e4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:51.117839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3cc4981b-3f37-4973-96e2-a77282c9e5e4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:51.118278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3cc4981b-3f37-4973-96e2-a77282c9e5e4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:51.119039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3cc4981b-3f37-4973-96e2-a77282c9e5e4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:51.119465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3cc4981b-3f37-4973-96e2-a77282c9e5e4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:51.250505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3cc4981b-3f37-4973-96e2-a77282c9e5e4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:51.258759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3cc4981b-3f37-4973-96e2-a77282c9e5e4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.152240 Jun 10 13:20:51.259333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 800/1560] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:51 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:51.263742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a2443df4-cda6-4909-ab9c-6ef685267f2c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:51.264336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a2443df4-cda6-4909-ab9c-6ef685267f2c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:51.270671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2443df4-cda6-4909-ab9c-6ef685267f2c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:51.271113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2443df4-cda6-4909-ab9c-6ef685267f2c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:51.271870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2443df4-cda6-4909-ab9c-6ef685267f2c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:51.272300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2443df4-cda6-4909-ab9c-6ef685267f2c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:51.420348 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a2443df4-cda6-4909-ab9c-6ef685267f2c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:51.428914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a2443df4-cda6-4909-ab9c-6ef685267f2c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.168579 Jun 10 13:20:51.429489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 801/1561] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:51 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:52.276253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e32a38fa-5abe-48ff-9e1f-5dcd143d4026 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:52.276884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e32a38fa-5abe-48ff-9e1f-5dcd143d4026 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:52.283361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e32a38fa-5abe-48ff-9e1f-5dcd143d4026 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:52.283799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e32a38fa-5abe-48ff-9e1f-5dcd143d4026 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:52.285223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e32a38fa-5abe-48ff-9e1f-5dcd143d4026 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:52.285653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e32a38fa-5abe-48ff-9e1f-5dcd143d4026 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:52.411859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e32a38fa-5abe-48ff-9e1f-5dcd143d4026 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:52.419248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e32a38fa-5abe-48ff-9e1f-5dcd143d4026 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146805 Jun 10 13:20:52.419810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 802/1562] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:52 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:52.445410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-60785d5c-6450-4c65-97fb-1c10137a55a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:52.446034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-60785d5c-6450-4c65-97fb-1c10137a55a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:52.451655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60785d5c-6450-4c65-97fb-1c10137a55a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:52.452096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60785d5c-6450-4c65-97fb-1c10137a55a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:52.452873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60785d5c-6450-4c65-97fb-1c10137a55a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:52.453305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60785d5c-6450-4c65-97fb-1c10137a55a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:52.584805 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-60785d5c-6450-4c65-97fb-1c10137a55a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:52.590996 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-60785d5c-6450-4c65-97fb-1c10137a55a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.149136 Jun 10 13:20:52.591561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 803/1563] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:52 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:53.435445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b6c04a2a-0d7e-4d25-9a13-4b924c34dbbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:53.436084 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b6c04a2a-0d7e-4d25-9a13-4b924c34dbbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:53.442637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b6c04a2a-0d7e-4d25-9a13-4b924c34dbbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:53.443082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b6c04a2a-0d7e-4d25-9a13-4b924c34dbbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:53.443836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b6c04a2a-0d7e-4d25-9a13-4b924c34dbbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:53.444267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b6c04a2a-0d7e-4d25-9a13-4b924c34dbbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:53.573333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b6c04a2a-0d7e-4d25-9a13-4b924c34dbbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:53.580704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b6c04a2a-0d7e-4d25-9a13-4b924c34dbbf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149157 Jun 10 13:20:53.581331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 804/1564] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:53 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:53.607245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f26eac0f-d848-45ff-9a8f-e65dbcc9f3a4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:53.607787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f26eac0f-d848-45ff-9a8f-e65dbcc9f3a4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:53.614214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f26eac0f-d848-45ff-9a8f-e65dbcc9f3a4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:53.614661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f26eac0f-d848-45ff-9a8f-e65dbcc9f3a4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:53.615413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f26eac0f-d848-45ff-9a8f-e65dbcc9f3a4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:53.615846 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f26eac0f-d848-45ff-9a8f-e65dbcc9f3a4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:53.633128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:20:53.746037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f26eac0f-d848-45ff-9a8f-e65dbcc9f3a4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:53.754320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f26eac0f-d848-45ff-9a8f-e65dbcc9f3a4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.150739 Jun 10 13:20:53.755248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 805/1565] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:53 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:54.597981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-31b217d9-f0a2-47b4-bd91-e3da0ccf1ea5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:54.598623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-31b217d9-f0a2-47b4-bd91-e3da0ccf1ea5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:54.605291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31b217d9-f0a2-47b4-bd91-e3da0ccf1ea5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:54.605739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31b217d9-f0a2-47b4-bd91-e3da0ccf1ea5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:54.606498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31b217d9-f0a2-47b4-bd91-e3da0ccf1ea5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:54.606930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31b217d9-f0a2-47b4-bd91-e3da0ccf1ea5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:54.741368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-31b217d9-f0a2-47b4-bd91-e3da0ccf1ea5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:54.748746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-31b217d9-f0a2-47b4-bd91-e3da0ccf1ea5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.154866 Jun 10 13:20:54.749332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 806/1566] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:54 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:20:54.771986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7a906299-9848-4161-9126-f006c20ee23a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:54.772615 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7a906299-9848-4161-9126-f006c20ee23a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:54.778606 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a906299-9848-4161-9126-f006c20ee23a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:54.779047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a906299-9848-4161-9126-f006c20ee23a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:54.779851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a906299-9848-4161-9126-f006c20ee23a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:54.780280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a906299-9848-4161-9126-f006c20ee23a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:54.904181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7a906299-9848-4161-9126-f006c20ee23a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:54.912650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7a906299-9848-4161-9126-f006c20ee23a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144264 Jun 10 13:20:54.913245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 807/1567] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:54 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:55.766298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-70cdf46e-41fa-4925-a0fb-34d6087166d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:55.766963 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-70cdf46e-41fa-4925-a0fb-34d6087166d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:55.773963 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-70cdf46e-41fa-4925-a0fb-34d6087166d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:55.774411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-70cdf46e-41fa-4925-a0fb-34d6087166d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:55.775165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-70cdf46e-41fa-4925-a0fb-34d6087166d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:55.775599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-70cdf46e-41fa-4925-a0fb-34d6087166d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:55.904012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-70cdf46e-41fa-4925-a0fb-34d6087166d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:55.911430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-70cdf46e-41fa-4925-a0fb-34d6087166d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.148961 Jun 10 13:20:55.911992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 808/1568] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:55 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:55.929681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d8b6ba7c-54cd-4260-9259-c0071440969c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:55.930309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d8b6ba7c-54cd-4260-9259-c0071440969c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:55.936143 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d8b6ba7c-54cd-4260-9259-c0071440969c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:55.936594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d8b6ba7c-54cd-4260-9259-c0071440969c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:55.937474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d8b6ba7c-54cd-4260-9259-c0071440969c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:55.937909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d8b6ba7c-54cd-4260-9259-c0071440969c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:56.063602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d8b6ba7c-54cd-4260-9259-c0071440969c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:56.072008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d8b6ba7c-54cd-4260-9259-c0071440969c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145773 Jun 10 13:20:56.072578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 809/1569] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:55 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:56.928957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-197aea78-b61e-4794-bf1d-90e0b75c6203 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:56.929623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-197aea78-b61e-4794-bf1d-90e0b75c6203 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:56.936042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-197aea78-b61e-4794-bf1d-90e0b75c6203 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:56.936494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-197aea78-b61e-4794-bf1d-90e0b75c6203 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:56.937286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-197aea78-b61e-4794-bf1d-90e0b75c6203 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:56.937717 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-197aea78-b61e-4794-bf1d-90e0b75c6203 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:57.062209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-197aea78-b61e-4794-bf1d-90e0b75c6203 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:57.069668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-197aea78-b61e-4794-bf1d-90e0b75c6203 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.145136 Jun 10 13:20:57.070168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 810/1570] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:56 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:57.088621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5750c415-54a7-4fa9-bc27-751044a07d74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:57.089288 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5750c415-54a7-4fa9-bc27-751044a07d74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:57.094946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5750c415-54a7-4fa9-bc27-751044a07d74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:57.095387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5750c415-54a7-4fa9-bc27-751044a07d74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:57.096131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5750c415-54a7-4fa9-bc27-751044a07d74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:57.096558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5750c415-54a7-4fa9-bc27-751044a07d74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:57.223485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5750c415-54a7-4fa9-bc27-751044a07d74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:57.231777 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5750c415-54a7-4fa9-bc27-751044a07d74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146705 Jun 10 13:20:57.232341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 811/1571] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:57 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:58.086926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-73d8f39a-22b4-4be4-a95f-463462795fb3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:58.087619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-73d8f39a-22b4-4be4-a95f-463462795fb3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:58.094104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-73d8f39a-22b4-4be4-a95f-463462795fb3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:58.094548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-73d8f39a-22b4-4be4-a95f-463462795fb3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:58.095303 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-73d8f39a-22b4-4be4-a95f-463462795fb3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:58.095732 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-73d8f39a-22b4-4be4-a95f-463462795fb3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:58.224609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-73d8f39a-22b4-4be4-a95f-463462795fb3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:58.232328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-73d8f39a-22b4-4be4-a95f-463462795fb3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149242 Jun 10 13:20:58.232907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 812/1572] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:58 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:58.249245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d1cc1759-7992-499b-9543-e2f29025b0d0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:58.249839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d1cc1759-7992-499b-9543-e2f29025b0d0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:58.256118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d1cc1759-7992-499b-9543-e2f29025b0d0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:58.256561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d1cc1759-7992-499b-9543-e2f29025b0d0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:58.257349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d1cc1759-7992-499b-9543-e2f29025b0d0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:58.257781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d1cc1759-7992-499b-9543-e2f29025b0d0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:58.384158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d1cc1759-7992-499b-9543-e2f29025b0d0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:58.392034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d1cc1759-7992-499b-9543-e2f29025b0d0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146308 Jun 10 13:20:58.392606 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 813/1573] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:58 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:59.248685 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-31696a9a-6473-4a34-89d8-3eba8c1bb4ea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:59.249383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-31696a9a-6473-4a34-89d8-3eba8c1bb4ea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:59.256328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31696a9a-6473-4a34-89d8-3eba8c1bb4ea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:59.256783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31696a9a-6473-4a34-89d8-3eba8c1bb4ea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:59.257548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31696a9a-6473-4a34-89d8-3eba8c1bb4ea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:59.257993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31696a9a-6473-4a34-89d8-3eba8c1bb4ea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:59.388410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-31696a9a-6473-4a34-89d8-3eba8c1bb4ea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:59.395791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-31696a9a-6473-4a34-89d8-3eba8c1bb4ea tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.150932 Jun 10 13:20:59.396629 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 814/1574] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:59 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:20:59.408104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d3a3fab8-89e4-4b1c-83cc-b517fd7b06a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:20:59.408694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d3a3fab8-89e4-4b1c-83cc-b517fd7b06a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:20:59.414746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d3a3fab8-89e4-4b1c-83cc-b517fd7b06a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:59.415182 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d3a3fab8-89e4-4b1c-83cc-b517fd7b06a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:59.415923 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d3a3fab8-89e4-4b1c-83cc-b517fd7b06a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:20:59.416349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d3a3fab8-89e4-4b1c-83cc-b517fd7b06a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:20:59.537565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d3a3fab8-89e4-4b1c-83cc-b517fd7b06a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:20:59.545576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d3a3fab8-89e4-4b1c-83cc-b517fd7b06a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.140779 Jun 10 13:20:59.546147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 815/1575] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:20:59 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:00.413202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-171c968b-6c4a-4523-9695-0766439a193e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:00.413808 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-171c968b-6c4a-4523-9695-0766439a193e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:00.420331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-171c968b-6c4a-4523-9695-0766439a193e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:00.420793 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-171c968b-6c4a-4523-9695-0766439a193e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:00.421559 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-171c968b-6c4a-4523-9695-0766439a193e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:00.421989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-171c968b-6c4a-4523-9695-0766439a193e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:00.551148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-171c968b-6c4a-4523-9695-0766439a193e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:00.558639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-171c968b-6c4a-4523-9695-0766439a193e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149943 Jun 10 13:21:00.559199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 816/1576] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:00 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:00.563650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7acbec94-6851-4e26-afef-72be6ce174f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:00.564250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7acbec94-6851-4e26-afef-72be6ce174f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:00.569832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7acbec94-6851-4e26-afef-72be6ce174f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:00.570273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7acbec94-6851-4e26-afef-72be6ce174f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:00.571055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7acbec94-6851-4e26-afef-72be6ce174f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:00.571485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7acbec94-6851-4e26-afef-72be6ce174f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:00.694989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7acbec94-6851-4e26-afef-72be6ce174f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:00.702883 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7acbec94-6851-4e26-afef-72be6ce174f2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.142696 Jun 10 13:21:00.703449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 817/1577] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:00 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:01.575566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9ec63c9c-37cf-4d44-a88b-e6213e6d3176 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:01.576226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9ec63c9c-37cf-4d44-a88b-e6213e6d3176 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:01.582280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ec63c9c-37cf-4d44-a88b-e6213e6d3176 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:01.582732 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ec63c9c-37cf-4d44-a88b-e6213e6d3176 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:01.583488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ec63c9c-37cf-4d44-a88b-e6213e6d3176 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:01.583921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ec63c9c-37cf-4d44-a88b-e6213e6d3176 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:01.711534 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9ec63c9c-37cf-4d44-a88b-e6213e6d3176 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:01.719667 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9ec63c9c-37cf-4d44-a88b-e6213e6d3176 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.147863 Jun 10 13:21:01.720237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 818/1578] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:01 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:01.725245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b9780cf9-8851-4836-bee3-509be9832f4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:01.725838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b9780cf9-8851-4836-bee3-509be9832f4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:01.731632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9780cf9-8851-4836-bee3-509be9832f4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:01.732071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9780cf9-8851-4836-bee3-509be9832f4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:01.732878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9780cf9-8851-4836-bee3-509be9832f4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:01.733306 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9780cf9-8851-4836-bee3-509be9832f4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:01.864687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b9780cf9-8851-4836-bee3-509be9832f4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:01.872445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b9780cf9-8851-4836-bee3-509be9832f4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.151285 Jun 10 13:21:01.873033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 819/1579] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:01 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:02.738424 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e098aa88-f596-4345-bc92-7728858760de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:02.739058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e098aa88-f596-4345-bc92-7728858760de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:02.745646 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e098aa88-f596-4345-bc92-7728858760de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:02.746097 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e098aa88-f596-4345-bc92-7728858760de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:02.746870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e098aa88-f596-4345-bc92-7728858760de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:02.747289 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e098aa88-f596-4345-bc92-7728858760de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:02.872436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e098aa88-f596-4345-bc92-7728858760de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:02.880501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e098aa88-f596-4345-bc92-7728858760de tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146493 Jun 10 13:21:02.881109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 820/1580] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:02 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:02.888733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-62a678d6-3707-4c39-9e17-f3f26ceb4a02 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:02.889358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-62a678d6-3707-4c39-9e17-f3f26ceb4a02 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:02.894976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62a678d6-3707-4c39-9e17-f3f26ceb4a02 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:02.895426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62a678d6-3707-4c39-9e17-f3f26ceb4a02 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:02.896180 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62a678d6-3707-4c39-9e17-f3f26ceb4a02 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:02.896615 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62a678d6-3707-4c39-9e17-f3f26ceb4a02 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:03.027350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-62a678d6-3707-4c39-9e17-f3f26ceb4a02 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:03.030985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-62a678d6-3707-4c39-9e17-f3f26ceb4a02 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145638 Jun 10 13:21:03.031553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 821/1581] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:02 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:03.898047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7b42be29-12c8-44d1-b704-cfea7a151ef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:03.898694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7b42be29-12c8-44d1-b704-cfea7a151ef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:03.905279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b42be29-12c8-44d1-b704-cfea7a151ef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:03.905728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b42be29-12c8-44d1-b704-cfea7a151ef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:03.906493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b42be29-12c8-44d1-b704-cfea7a151ef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:03.906922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b42be29-12c8-44d1-b704-cfea7a151ef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:03.910877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:21:04.054090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7b42be29-12c8-44d1-b704-cfea7a151ef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:04.061585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7b42be29-12c8-44d1-b704-cfea7a151ef6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.167998 Jun 10 13:21:04.062161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 822/1582] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:03 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:04.066985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f627e747-82fe-4ac6-bb7a-5bae4b1d8e74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:04.067591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f627e747-82fe-4ac6-bb7a-5bae4b1d8e74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:04.073493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f627e747-82fe-4ac6-bb7a-5bae4b1d8e74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:04.073941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f627e747-82fe-4ac6-bb7a-5bae4b1d8e74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:04.074752 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f627e747-82fe-4ac6-bb7a-5bae4b1d8e74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:04.075176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f627e747-82fe-4ac6-bb7a-5bae4b1d8e74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:04.380954 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f627e747-82fe-4ac6-bb7a-5bae4b1d8e74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:04.388759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f627e747-82fe-4ac6-bb7a-5bae4b1d8e74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.325666 Jun 10 13:21:04.389335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 823/1583] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:04 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:05.079160 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7b216407-18ad-43f9-8080-e6f4829fc118 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:05.079847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7b216407-18ad-43f9-8080-e6f4829fc118 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:05.086263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b216407-18ad-43f9-8080-e6f4829fc118 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:05.086707 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b216407-18ad-43f9-8080-e6f4829fc118 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:05.087459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b216407-18ad-43f9-8080-e6f4829fc118 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:05.087890 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b216407-18ad-43f9-8080-e6f4829fc118 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:05.211670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7b216407-18ad-43f9-8080-e6f4829fc118 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:05.219129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7b216407-18ad-43f9-8080-e6f4829fc118 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.143829 Jun 10 13:21:05.219691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 824/1584] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:05 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:05.405595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-71ead5b6-bf02-4e23-9115-ecd063c7e3ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:05.406191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-71ead5b6-bf02-4e23-9115-ecd063c7e3ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:05.412586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71ead5b6-bf02-4e23-9115-ecd063c7e3ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:05.413062 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71ead5b6-bf02-4e23-9115-ecd063c7e3ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:05.413807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71ead5b6-bf02-4e23-9115-ecd063c7e3ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:05.414234 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71ead5b6-bf02-4e23-9115-ecd063c7e3ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:05.539812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-71ead5b6-bf02-4e23-9115-ecd063c7e3ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:05.547586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-71ead5b6-bf02-4e23-9115-ecd063c7e3ef tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145811 Jun 10 13:21:05.548145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 825/1585] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:05 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:06.236319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9e9e3ce4-7986-4551-b92a-088430fc945a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:06.236991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9e9e3ce4-7986-4551-b92a-088430fc945a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:06.243433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e9e3ce4-7986-4551-b92a-088430fc945a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:06.243878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e9e3ce4-7986-4551-b92a-088430fc945a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:06.244630 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e9e3ce4-7986-4551-b92a-088430fc945a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:06.245086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e9e3ce4-7986-4551-b92a-088430fc945a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:06.374555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9e9e3ce4-7986-4551-b92a-088430fc945a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:06.382576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9e9e3ce4-7986-4551-b92a-088430fc945a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.150114 Jun 10 13:21:06.383146 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 826/1586] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:06 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:06.564616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1ab6ae5b-6b9d-4acd-aabc-fd08965b14a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:06.565787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1ab6ae5b-6b9d-4acd-aabc-fd08965b14a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:06.571501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ab6ae5b-6b9d-4acd-aabc-fd08965b14a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:06.571947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ab6ae5b-6b9d-4acd-aabc-fd08965b14a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:06.572698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ab6ae5b-6b9d-4acd-aabc-fd08965b14a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:06.573157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ab6ae5b-6b9d-4acd-aabc-fd08965b14a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:06.697484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1ab6ae5b-6b9d-4acd-aabc-fd08965b14a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:06.705258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1ab6ae5b-6b9d-4acd-aabc-fd08965b14a2 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144251 Jun 10 13:21:06.705820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 827/1587] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:06 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:07.399460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-abc472fc-da36-46ba-a140-36cb5a4a5f2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:07.400106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-abc472fc-da36-46ba-a140-36cb5a4a5f2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:07.406532 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-abc472fc-da36-46ba-a140-36cb5a4a5f2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:07.406970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-abc472fc-da36-46ba-a140-36cb5a4a5f2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:07.407707 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-abc472fc-da36-46ba-a140-36cb5a4a5f2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:07.408135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-abc472fc-da36-46ba-a140-36cb5a4a5f2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:07.535522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-abc472fc-da36-46ba-a140-36cb5a4a5f2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:07.542865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-abc472fc-da36-46ba-a140-36cb5a4a5f2d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.147287 Jun 10 13:21:07.543417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 828/1588] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:07 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:07.721199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-97e154d1-097d-4983-a297-77c0f3967c39 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:07.721798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-97e154d1-097d-4983-a297-77c0f3967c39 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:07.727556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97e154d1-097d-4983-a297-77c0f3967c39 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:07.728260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97e154d1-097d-4983-a297-77c0f3967c39 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:07.729030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97e154d1-097d-4983-a297-77c0f3967c39 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:07.729460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97e154d1-097d-4983-a297-77c0f3967c39 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:07.854088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-97e154d1-097d-4983-a297-77c0f3967c39 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:07.862451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-97e154d1-097d-4983-a297-77c0f3967c39 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144288 Jun 10 13:21:07.863009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 829/1589] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:07 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:08.560373 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-821fe74c-f175-493e-aa80-feeaca057df1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:08.560994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-821fe74c-f175-493e-aa80-feeaca057df1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:08.567524 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-821fe74c-f175-493e-aa80-feeaca057df1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:08.567968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-821fe74c-f175-493e-aa80-feeaca057df1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:08.568727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-821fe74c-f175-493e-aa80-feeaca057df1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:08.569192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-821fe74c-f175-493e-aa80-feeaca057df1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:08.703154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-821fe74c-f175-493e-aa80-feeaca057df1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:08.709562 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-821fe74c-f175-493e-aa80-feeaca057df1 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.153088 Jun 10 13:21:08.710139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 830/1590] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:08 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:08.879400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-23696070-059f-4e98-ab0b-5ac365d931c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:08.879926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-23696070-059f-4e98-ab0b-5ac365d931c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:08.886212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-23696070-059f-4e98-ab0b-5ac365d931c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:08.888159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-23696070-059f-4e98-ab0b-5ac365d931c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:08.888159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-23696070-059f-4e98-ab0b-5ac365d931c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:08.888159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-23696070-059f-4e98-ab0b-5ac365d931c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:09.019998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-23696070-059f-4e98-ab0b-5ac365d931c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:09.027857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-23696070-059f-4e98-ab0b-5ac365d931c1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.152362 Jun 10 13:21:09.028419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 831/1591] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:08 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:09.727481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1da38ec6-5ce1-4878-af6f-8d1f6d773771 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:09.728376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1da38ec6-5ce1-4878-af6f-8d1f6d773771 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:09.734836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1da38ec6-5ce1-4878-af6f-8d1f6d773771 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:09.735276 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1da38ec6-5ce1-4878-af6f-8d1f6d773771 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:09.736018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1da38ec6-5ce1-4878-af6f-8d1f6d773771 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:09.736445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1da38ec6-5ce1-4878-af6f-8d1f6d773771 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:09.862341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1da38ec6-5ce1-4878-af6f-8d1f6d773771 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:09.869780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1da38ec6-5ce1-4878-af6f-8d1f6d773771 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146280 Jun 10 13:21:09.870336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 832/1592] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:09 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:10.044867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f8c12ab3-a6c2-45f8-9742-59b6839b2620 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:10.045402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f8c12ab3-a6c2-45f8-9742-59b6839b2620 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:10.051979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8c12ab3-a6c2-45f8-9742-59b6839b2620 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:10.052421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8c12ab3-a6c2-45f8-9742-59b6839b2620 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:10.053200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8c12ab3-a6c2-45f8-9742-59b6839b2620 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:10.053628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8c12ab3-a6c2-45f8-9742-59b6839b2620 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:10.183188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f8c12ab3-a6c2-45f8-9742-59b6839b2620 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:10.190992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f8c12ab3-a6c2-45f8-9742-59b6839b2620 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.150114 Jun 10 13:21:10.191555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 833/1593] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:10 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:10.886866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d30229bd-93e8-4edc-8956-21d8586d69b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:10.887529 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d30229bd-93e8-4edc-8956-21d8586d69b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:10.893945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d30229bd-93e8-4edc-8956-21d8586d69b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:10.894390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d30229bd-93e8-4edc-8956-21d8586d69b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:10.895141 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d30229bd-93e8-4edc-8956-21d8586d69b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:10.895571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d30229bd-93e8-4edc-8956-21d8586d69b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:11.022082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d30229bd-93e8-4edc-8956-21d8586d69b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:11.029449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d30229bd-93e8-4edc-8956-21d8586d69b0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.146466 Jun 10 13:21:11.030013 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 834/1594] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:10 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:11.207689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d4713424-3604-4275-815e-630f9111d9d5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:11.208679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d4713424-3604-4275-815e-630f9111d9d5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:11.214323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4713424-3604-4275-815e-630f9111d9d5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:11.214774 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4713424-3604-4275-815e-630f9111d9d5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:11.215531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4713424-3604-4275-815e-630f9111d9d5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:11.216012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4713424-3604-4275-815e-630f9111d9d5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:11.344875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d4713424-3604-4275-815e-630f9111d9d5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:11.353498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d4713424-3604-4275-815e-630f9111d9d5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.149741 Jun 10 13:21:11.354100 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 835/1595] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:11 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:12.046023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5e800693-c50c-46aa-8c79-57a73ecaef33 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:12.046698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5e800693-c50c-46aa-8c79-57a73ecaef33 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:12.053755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e800693-c50c-46aa-8c79-57a73ecaef33 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:12.054204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e800693-c50c-46aa-8c79-57a73ecaef33 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:12.054967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e800693-c50c-46aa-8c79-57a73ecaef33 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:12.055404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e800693-c50c-46aa-8c79-57a73ecaef33 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:12.184069 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5e800693-c50c-46aa-8c79-57a73ecaef33 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:12.191571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5e800693-c50c-46aa-8c79-57a73ecaef33 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.149451 Jun 10 13:21:12.192130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 836/1596] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:12 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:12.371037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-256c8329-13d6-4ee8-9a9d-27d44dae036b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:12.371641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-256c8329-13d6-4ee8-9a9d-27d44dae036b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:12.377656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-256c8329-13d6-4ee8-9a9d-27d44dae036b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:12.378105 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-256c8329-13d6-4ee8-9a9d-27d44dae036b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:12.378861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-256c8329-13d6-4ee8-9a9d-27d44dae036b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:12.379298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-256c8329-13d6-4ee8-9a9d-27d44dae036b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:12.504165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-256c8329-13d6-4ee8-9a9d-27d44dae036b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:12.512027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-256c8329-13d6-4ee8-9a9d-27d44dae036b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144828 Jun 10 13:21:12.512589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 837/1597] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:12 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:13.208403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-aadd6370-3db3-4d45-9ae4-a29b40690978 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:13.209100 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-aadd6370-3db3-4d45-9ae4-a29b40690978 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:13.212429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:21:13.215758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aadd6370-3db3-4d45-9ae4-a29b40690978 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:13.216198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aadd6370-3db3-4d45-9ae4-a29b40690978 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:13.216970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aadd6370-3db3-4d45-9ae4-a29b40690978 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:13.217398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aadd6370-3db3-4d45-9ae4-a29b40690978 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:13.344563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-aadd6370-3db3-4d45-9ae4-a29b40690978 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:13.348047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:21:13.352355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-aadd6370-3db3-4d45-9ae4-a29b40690978 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.147803 Jun 10 13:21:13.353206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 838/1598] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:13 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:13.528464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5680f395-51f6-452b-94da-fc97939fc783 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:13.529023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5680f395-51f6-452b-94da-fc97939fc783 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:13.535774 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5680f395-51f6-452b-94da-fc97939fc783 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:13.536292 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5680f395-51f6-452b-94da-fc97939fc783 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:13.537020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5680f395-51f6-452b-94da-fc97939fc783 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:13.537452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5680f395-51f6-452b-94da-fc97939fc783 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:13.661072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5680f395-51f6-452b-94da-fc97939fc783 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:13.668819 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5680f395-51f6-452b-94da-fc97939fc783 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144001 Jun 10 13:21:13.669378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 839/1599] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:13 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:14.370141 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3933939d-e156-4d0e-a303-172aca588131 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:14.370747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3933939d-e156-4d0e-a303-172aca588131 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:14.377764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3933939d-e156-4d0e-a303-172aca588131 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:14.378209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3933939d-e156-4d0e-a303-172aca588131 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:14.378962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3933939d-e156-4d0e-a303-172aca588131 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:14.379421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3933939d-e156-4d0e-a303-172aca588131 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:14.506984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3933939d-e156-4d0e-a303-172aca588131 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:14.514296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3933939d-e156-4d0e-a303-172aca588131 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.148003 Jun 10 13:21:14.514859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 840/1600] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:14 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:14.686294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1a032cef-9a85-4a06-a783-66cc9d4d5ef5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:14.686904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1a032cef-9a85-4a06-a783-66cc9d4d5ef5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:14.693496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a032cef-9a85-4a06-a783-66cc9d4d5ef5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:14.693945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a032cef-9a85-4a06-a783-66cc9d4d5ef5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:14.694698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a032cef-9a85-4a06-a783-66cc9d4d5ef5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:14.695131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a032cef-9a85-4a06-a783-66cc9d4d5ef5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:14.819506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1a032cef-9a85-4a06-a783-66cc9d4d5ef5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:14.827295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1a032cef-9a85-4a06-a783-66cc9d4d5ef5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144852 Jun 10 13:21:14.833927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 841/1601] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:14 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:15.415752 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2f5dfd2d-f217-44b4-834b-ae1b45d17621 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:15.416364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2f5dfd2d-f217-44b4-834b-ae1b45d17621 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:15.422797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f5dfd2d-f217-44b4-834b-ae1b45d17621 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:15.423243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f5dfd2d-f217-44b4-834b-ae1b45d17621 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:15.423993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f5dfd2d-f217-44b4-834b-ae1b45d17621 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:15.424423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f5dfd2d-f217-44b4-834b-ae1b45d17621 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:18.711323 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.objects.instance [None req-3ca516c7-e8dc-45f7-a254-edb015bc43b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lazy-loading 'services' on Instance uuid a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 {{(pid=118093) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:21:18.948501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-2f5dfd2d-f217-44b4-834b-ae1b45d17621 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lazy-loading 'services' on Instance uuid a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:21:19.071801 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.wsgi [None req-3ca516c7-e8dc-45f7-a254-edb015bc43b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cff9c194-88b3-4b10-b670-2415fcf7b16a) Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3ca516c7-e8dc-45f7-a254-edb015bc43b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Request method failure captured: Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: request: DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e HTTP/1.1 Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept: application/json Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Accept-Encoding: identity Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Connection: close Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Length: 0 Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Content-Type: application/json Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Host: 10.222.0.25 Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Openstack-System-Scope: None Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: User-Agent: python-urllib3/1.26.9 Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Auth-Token: gAAAAABio0HlyICQX9l5KFYyUsFzMtUgTAzPRfxQzNN7dM9kV13g0iWQTn9QMENYtfZqE20ETOnCUUxvj1yudvSxd1bFTvMSIvc_eVAfoGvb2Phk1I--eUGA8-ePhD-Rp01rKEhzSDpYm-ppUB1HJXel5mUDpZwxED5NTdU7SM0nV7mlEGF1wbo Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Id: None Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Domain-Name: None Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Identity-Status: Confirmed Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Is-Admin-Project: True Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Id: default Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Domain-Name: Default Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Id: ff2616337ae440a89047fd9a50218387 Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Role: reader,member Jun 10 13:21:19.073988 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Roles: reader,member Jun 10 13:21:19.075614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ff2616337ae440a89047fd9a50218387"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ff2616337ae440a89047fd9a50218387"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ff2616337ae440a89047fd9a50218387", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ff2616337ae440a89047fd9a50218387"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:21:19.075614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:21:19.075614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Id: ff2616337ae440a89047fd9a50218387 Jun 10 13:21:19.075614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:21:19.075614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User: tempest-VolumesSnapshotTestJSON-1698707508-project Jun 10 13:21:19.075614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Id: default Jun 10 13:21:19.075614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Domain-Name: Default Jun 10 13:21:19.075614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Id: 849e36f8653d4d1f9a7fb1df8fb913a9 Jun 10 13:21:19.075614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: X-User-Name: tempest-VolumesSnapshotTestJSON-1698707508-project Jun 10 13:21:19.075614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: method: > Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cff9c194-88b3-4b10-b670-2415fcf7b16a) Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: action_args: {'server_id': 'a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744', 'id': '802588c0-10d5-4ca0-8cbb-585dee61c94e'} Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jun 10 13:21:19.076701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cff9c194-88b3-4b10-b670-2415fcf7b16a) Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:21:19.077961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cff9c194-88b3-4b10-b670-2415fcf7b16a) Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:21:19.079184 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cff9c194-88b3-4b10-b670-2415fcf7b16a) Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:21:19.081115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cff9c194-88b3-4b10-b670-2415fcf7b16a) Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cff9c194-88b3-4b10-b670-2415fcf7b16a) Jun 10 13:21:19.083884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.085149 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3ca516c7-e8dc-45f7-a254-edb015bc43b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cff9c194-88b3-4b10-b670-2415fcf7b16a) {{(pid=118093) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:21:19.085149 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3ca516c7-e8dc-45f7-a254-edb015bc43b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e" status: 400 len: 269 microversion: 2.1 time: 63.006139 Jun 10 13:21:19.085149 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 761/1602] 10.222.0.25 () {58 vars in 1415 bytes} [Fri Jun 10 13:20:16 2022] DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e => generated 269 bytes in 63007 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jun 10 13:21:19.085149 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-669b0f53-6919-40c1-95ee-71f0fc7b16a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:19.085518 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-669b0f53-6919-40c1-95ee-71f0fc7b16a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:19.089112 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-669b0f53-6919-40c1-95ee-71f0fc7b16a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:19.089559 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-669b0f53-6919-40c1-95ee-71f0fc7b16a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:19.090328 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-669b0f53-6919-40c1-95ee-71f0fc7b16a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:19.090763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-669b0f53-6919-40c1-95ee-71f0fc7b16a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:19.113904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-2f5dfd2d-f217-44b4-834b-ae1b45d17621 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d84e6845-0eba-4f33-bf14-6250161ce54d) Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2f5dfd2d-f217-44b4-834b-ae1b45d17621 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Request method failure captured: Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e HTTP/1.1 Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0HlyICQX9l5KFYyUsFzMtUgTAzPRfxQzNN7dM9kV13g0iWQTn9QMENYtfZqE20ETOnCUUxvj1yudvSxd1bFTvMSIvc_eVAfoGvb2Phk1I--eUGA8-ePhD-Rp01rKEhzSDpYm-ppUB1HJXel5mUDpZwxED5NTdU7SM0nV7mlEGF1wbo Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: ff2616337ae440a89047fd9a50218387 Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:21:19.116247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:21:19.117921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ff2616337ae440a89047fd9a50218387"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ff2616337ae440a89047fd9a50218387"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ff2616337ae440a89047fd9a50218387", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ff2616337ae440a89047fd9a50218387"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:21:19.117921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:21:19.117921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: ff2616337ae440a89047fd9a50218387 Jun 10 13:21:19.117921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1698707508 Jun 10 13:21:19.117921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesSnapshotTestJSON-1698707508-project Jun 10 13:21:19.117921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:21:19.117921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:21:19.117921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 849e36f8653d4d1f9a7fb1df8fb913a9 Jun 10 13:21:19.117921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesSnapshotTestJSON-1698707508-project Jun 10 13:21:19.117921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d84e6845-0eba-4f33-bf14-6250161ce54d) Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'server_id': 'a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744', 'id': '802588c0-10d5-4ca0-8cbb-585dee61c94e'} Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jun 10 13:21:19.118640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d84e6845-0eba-4f33-bf14-6250161ce54d) Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:21:19.119885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d84e6845-0eba-4f33-bf14-6250161ce54d) Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:21:19.121488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d84e6845-0eba-4f33-bf14-6250161ce54d) Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:21:19.122772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d84e6845-0eba-4f33-bf14-6250161ce54d) Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d84e6845-0eba-4f33-bf14-6250161ce54d) Jun 10 13:21:19.124173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:21:19.137865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2f5dfd2d-f217-44b4-834b-ae1b45d17621 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d84e6845-0eba-4f33-bf14-6250161ce54d) {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:21:19.138760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2f5dfd2d-f217-44b4-834b-ae1b45d17621 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e" status: 400 len: 269 microversion: 2.1 time: 3.727000 Jun 10 13:21:19.139365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 842/1603] 10.222.0.25 () {58 vars in 1415 bytes} [Fri Jun 10 13:21:15 2022] DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744/os-volume_attachments/802588c0-10d5-4ca0-8cbb-585dee61c94e => generated 269 bytes in 3728 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jun 10 13:21:19.157168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cfcaabd1-5dec-4343-a564-5d76cdfb0135 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:19.163553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cfcaabd1-5dec-4343-a564-5d76cdfb0135 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:19.190787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfcaabd1-5dec-4343-a564-5d76cdfb0135 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:19.190787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfcaabd1-5dec-4343-a564-5d76cdfb0135 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:19.190787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfcaabd1-5dec-4343-a564-5d76cdfb0135 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:19.190787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfcaabd1-5dec-4343-a564-5d76cdfb0135 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:19.335593 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-669b0f53-6919-40c1-95ee-71f0fc7b16a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:19.347404 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-669b0f53-6919-40c1-95ee-71f0fc7b16a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1451 microversion: 2.1 time: 0.267246 Jun 10 13:21:19.347404 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 762/1604] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:19 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1451 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:19.347656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cfcaabd1-5dec-4343-a564-5d76cdfb0135 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:19.355764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cfcaabd1-5dec-4343-a564-5d76cdfb0135 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.209206 Jun 10 13:21:19.356331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 843/1605] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:19 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:19.709021 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-2c8736b1-9116-4527-b242-1956c1cbed5e req-d1d3a375-9515-4f26-b879-8387b3237487 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3baaefc5-1a36-4fa6-88d9-1764e5631a32", "name": "network-vif-deleted", "tag": "72eeacbd-3288-4d28-90e4-93e208b2d17d"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:21:19.717160 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-2c8736b1-9116-4527-b242-1956c1cbed5e req-d1d3a375-9515-4f26-b879-8387b3237487 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:19.717776 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-2c8736b1-9116-4527-b242-1956c1cbed5e req-d1d3a375-9515-4f26-b879-8387b3237487 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:19.732065 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-2c8736b1-9116-4527-b242-1956c1cbed5e req-d1d3a375-9515-4f26-b879-8387b3237487 service nova] Creating event network-vif-deleted:72eeacbd-3288-4d28-90e4-93e208b2d17d for instance 3baaefc5-1a36-4fa6-88d9-1764e5631a32 on hv-cinder-maste Jun 10 13:21:19.739714 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-2c8736b1-9116-4527-b242-1956c1cbed5e req-d1d3a375-9515-4f26-b879-8387b3237487 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034317 Jun 10 13:21:19.740336 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 763/1606] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:21:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:21:20.363281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-432da34d-6bc6-4891-9e12-b0cdc443c4ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:20.363923 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-432da34d-6bc6-4891-9e12-b0cdc443c4ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:20.370625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-432da34d-6bc6-4891-9e12-b0cdc443c4ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:20.371065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-432da34d-6bc6-4891-9e12-b0cdc443c4ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:20.371851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-432da34d-6bc6-4891-9e12-b0cdc443c4ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:20.372644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-432da34d-6bc6-4891-9e12-b0cdc443c4ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:20.373517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7bb9be4f-6305-4dbf-9f36-c9d106bd104f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:20.374208 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7bb9be4f-6305-4dbf-9f36-c9d106bd104f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:20.380912 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7bb9be4f-6305-4dbf-9f36-c9d106bd104f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:20.381358 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7bb9be4f-6305-4dbf-9f36-c9d106bd104f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:20.382121 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7bb9be4f-6305-4dbf-9f36-c9d106bd104f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:20.382551 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7bb9be4f-6305-4dbf-9f36-c9d106bd104f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:20.518664 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7bb9be4f-6305-4dbf-9f36-c9d106bd104f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:20.526815 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7bb9be4f-6305-4dbf-9f36-c9d106bd104f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.157344 Jun 10 13:21:20.527386 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 764/1607] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:20 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:20.582868 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-432da34d-6bc6-4891-9e12-b0cdc443c4ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:20.590127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-432da34d-6bc6-4891-9e12-b0cdc443c4ce tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.231358 Jun 10 13:21:20.590670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 844/1608] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:20 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:21.544615 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6e998da4-43cf-4d60-914c-3ea6e0f69906 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:21.545343 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6e998da4-43cf-4d60-914c-3ea6e0f69906 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:21.551682 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e998da4-43cf-4d60-914c-3ea6e0f69906 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:21.552126 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e998da4-43cf-4d60-914c-3ea6e0f69906 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:21.552907 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e998da4-43cf-4d60-914c-3ea6e0f69906 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:21.553338 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e998da4-43cf-4d60-914c-3ea6e0f69906 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:21.607535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1b8ae89b-b58b-4498-b2f5-e43562ad800f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:21.608227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1b8ae89b-b58b-4498-b2f5-e43562ad800f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:21.614376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b8ae89b-b58b-4498-b2f5-e43562ad800f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:21.614827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b8ae89b-b58b-4498-b2f5-e43562ad800f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:21.615593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b8ae89b-b58b-4498-b2f5-e43562ad800f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:21.616022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b8ae89b-b58b-4498-b2f5-e43562ad800f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:21.677555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1b8ae89b-b58b-4498-b2f5-e43562ad800f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:21.685394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1b8ae89b-b58b-4498-b2f5-e43562ad800f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.081517 Jun 10 13:21:21.685918 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 845/1609] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:21 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:21.783176 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6e998da4-43cf-4d60-914c-3ea6e0f69906 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:21.791108 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6e998da4-43cf-4d60-914c-3ea6e0f69906 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.250331 Jun 10 13:21:21.791713 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 765/1610] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:21 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:22.702306 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1baba35a-da4a-4f45-8925-2a5cec16c459 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:22.702912 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1baba35a-da4a-4f45-8925-2a5cec16c459 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:22.709442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1baba35a-da4a-4f45-8925-2a5cec16c459 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:22.709889 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1baba35a-da4a-4f45-8925-2a5cec16c459 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:22.710644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1baba35a-da4a-4f45-8925-2a5cec16c459 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:22.711075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1baba35a-da4a-4f45-8925-2a5cec16c459 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:22.769749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1baba35a-da4a-4f45-8925-2a5cec16c459 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:22.776982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1baba35a-da4a-4f45-8925-2a5cec16c459 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.078509 Jun 10 13:21:22.777534 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 846/1611] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:22 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:22.808119 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b5a6add4-dede-473d-9fde-a20a5adbe5aa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:22.808732 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b5a6add4-dede-473d-9fde-a20a5adbe5aa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:22.815146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b5a6add4-dede-473d-9fde-a20a5adbe5aa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:22.815588 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b5a6add4-dede-473d-9fde-a20a5adbe5aa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:22.816345 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b5a6add4-dede-473d-9fde-a20a5adbe5aa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:22.816821 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b5a6add4-dede-473d-9fde-a20a5adbe5aa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:22.944973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b5a6add4-dede-473d-9fde-a20a5adbe5aa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:22.952934 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b5a6add4-dede-473d-9fde-a20a5adbe5aa tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148562 Jun 10 13:21:22.953429 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 766/1612] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:22 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:23.794993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-197790a9-fc59-4cbf-be1a-b1048b5d47a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:23.795598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-197790a9-fc59-4cbf-be1a-b1048b5d47a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:23.802137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-197790a9-fc59-4cbf-be1a-b1048b5d47a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:23.802612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-197790a9-fc59-4cbf-be1a-b1048b5d47a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:23.803365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-197790a9-fc59-4cbf-be1a-b1048b5d47a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:23.803797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-197790a9-fc59-4cbf-be1a-b1048b5d47a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:23.865122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-197790a9-fc59-4cbf-be1a-b1048b5d47a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:23.872477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-197790a9-fc59-4cbf-be1a-b1048b5d47a6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.081352 Jun 10 13:21:23.873052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 847/1613] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:23 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:23.971160 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d7372d7f-6ac0-48cd-b04f-0b89a2c4e78d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:23.971790 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d7372d7f-6ac0-48cd-b04f-0b89a2c4e78d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:23.978674 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d7372d7f-6ac0-48cd-b04f-0b89a2c4e78d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:23.979117 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d7372d7f-6ac0-48cd-b04f-0b89a2c4e78d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:23.980580 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d7372d7f-6ac0-48cd-b04f-0b89a2c4e78d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:23.981051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d7372d7f-6ac0-48cd-b04f-0b89a2c4e78d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:24.109674 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d7372d7f-6ac0-48cd-b04f-0b89a2c4e78d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:24.117717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d7372d7f-6ac0-48cd-b04f-0b89a2c4e78d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.151320 Jun 10 13:21:24.118277 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 767/1614] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:23 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:24.889104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ba66cbab-7907-4f77-b521-78a0cef71926 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:24.889719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ba66cbab-7907-4f77-b521-78a0cef71926 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:24.896877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba66cbab-7907-4f77-b521-78a0cef71926 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:24.897325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba66cbab-7907-4f77-b521-78a0cef71926 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:24.898086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba66cbab-7907-4f77-b521-78a0cef71926 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:24.898516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba66cbab-7907-4f77-b521-78a0cef71926 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:24.957204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ba66cbab-7907-4f77-b521-78a0cef71926 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:24.964554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ba66cbab-7907-4f77-b521-78a0cef71926 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079313 Jun 10 13:21:24.965142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 848/1615] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:24 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:25.135884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d3075f89-f893-4e60-ad46-345b2980fff4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:25.136501 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d3075f89-f893-4e60-ad46-345b2980fff4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:25.143005 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d3075f89-f893-4e60-ad46-345b2980fff4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:25.143403 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d3075f89-f893-4e60-ad46-345b2980fff4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:25.144166 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d3075f89-f893-4e60-ad46-345b2980fff4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:25.144600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d3075f89-f893-4e60-ad46-345b2980fff4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:25.267001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d3075f89-f893-4e60-ad46-345b2980fff4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:25.274855 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d3075f89-f893-4e60-ad46-345b2980fff4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.142791 Jun 10 13:21:25.275427 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 768/1616] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:25 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:25.981698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3a06eb00-eff7-4a58-b07c-6a920623bcf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:25.982307 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3a06eb00-eff7-4a58-b07c-6a920623bcf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:25.988857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3a06eb00-eff7-4a58-b07c-6a920623bcf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:25.989305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3a06eb00-eff7-4a58-b07c-6a920623bcf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:25.990065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3a06eb00-eff7-4a58-b07c-6a920623bcf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:25.990499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3a06eb00-eff7-4a58-b07c-6a920623bcf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:26.049040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3a06eb00-eff7-4a58-b07c-6a920623bcf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:26.056387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3a06eb00-eff7-4a58-b07c-6a920623bcf4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.078820 Jun 10 13:21:26.056969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 849/1617] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:25 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:26.292687 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-038d3567-f1e9-4b73-9905-4604a6cb7c9e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:26.293325 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-038d3567-f1e9-4b73-9905-4604a6cb7c9e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:26.299874 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-038d3567-f1e9-4b73-9905-4604a6cb7c9e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:26.300310 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-038d3567-f1e9-4b73-9905-4604a6cb7c9e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:26.301092 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-038d3567-f1e9-4b73-9905-4604a6cb7c9e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:26.301517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-038d3567-f1e9-4b73-9905-4604a6cb7c9e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:26.430889 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-038d3567-f1e9-4b73-9905-4604a6cb7c9e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:26.438668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-038d3567-f1e9-4b73-9905-4604a6cb7c9e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.149827 Jun 10 13:21:26.439763 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 769/1618] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:26 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:27.072861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4f6be452-20fa-4893-8fdb-b7604965aa40 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:27.073467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4f6be452-20fa-4893-8fdb-b7604965aa40 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:27.079915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4f6be452-20fa-4893-8fdb-b7604965aa40 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:27.080359 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4f6be452-20fa-4893-8fdb-b7604965aa40 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:27.081148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4f6be452-20fa-4893-8fdb-b7604965aa40 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:27.081579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4f6be452-20fa-4893-8fdb-b7604965aa40 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:27.143521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4f6be452-20fa-4893-8fdb-b7604965aa40 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:27.150931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4f6be452-20fa-4893-8fdb-b7604965aa40 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.081999 Jun 10 13:21:27.151489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 850/1619] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:27 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:27.455951 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b04ef8cc-7a6b-47a1-822a-461287bf0fce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:27.456555 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b04ef8cc-7a6b-47a1-822a-461287bf0fce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:27.463071 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b04ef8cc-7a6b-47a1-822a-461287bf0fce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:27.463517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b04ef8cc-7a6b-47a1-822a-461287bf0fce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:27.464269 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b04ef8cc-7a6b-47a1-822a-461287bf0fce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:27.464705 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b04ef8cc-7a6b-47a1-822a-461287bf0fce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:27.588285 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b04ef8cc-7a6b-47a1-822a-461287bf0fce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:27.596655 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b04ef8cc-7a6b-47a1-822a-461287bf0fce tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145130 Jun 10 13:21:27.597291 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 770/1620] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:27 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:28.168177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2ca0b8e3-0a24-440a-81f3-006911ecf02f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:28.168847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2ca0b8e3-0a24-440a-81f3-006911ecf02f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:28.175254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ca0b8e3-0a24-440a-81f3-006911ecf02f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:28.176357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ca0b8e3-0a24-440a-81f3-006911ecf02f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:28.176464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ca0b8e3-0a24-440a-81f3-006911ecf02f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:28.177464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ca0b8e3-0a24-440a-81f3-006911ecf02f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:28.239946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2ca0b8e3-0a24-440a-81f3-006911ecf02f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:28.247356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2ca0b8e3-0a24-440a-81f3-006911ecf02f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.083045 Jun 10 13:21:28.247911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 851/1621] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:28 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:28.614492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2d4b581b-6bd6-4b05-aa54-a51de086bb5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:28.615103 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2d4b581b-6bd6-4b05-aa54-a51de086bb5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:28.621650 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d4b581b-6bd6-4b05-aa54-a51de086bb5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:28.622097 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d4b581b-6bd6-4b05-aa54-a51de086bb5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:28.622858 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d4b581b-6bd6-4b05-aa54-a51de086bb5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:28.623290 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d4b581b-6bd6-4b05-aa54-a51de086bb5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:28.755230 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2d4b581b-6bd6-4b05-aa54-a51de086bb5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:28.763197 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2d4b581b-6bd6-4b05-aa54-a51de086bb5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.153149 Jun 10 13:21:28.767517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 771/1622] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:28 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:29.264049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-446ba886-ad4f-48c4-81f3-21f5e2da0de6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:29.264652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-446ba886-ad4f-48c4-81f3-21f5e2da0de6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:29.271837 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-446ba886-ad4f-48c4-81f3-21f5e2da0de6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:29.272285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-446ba886-ad4f-48c4-81f3-21f5e2da0de6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:29.273066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-446ba886-ad4f-48c4-81f3-21f5e2da0de6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:29.273491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-446ba886-ad4f-48c4-81f3-21f5e2da0de6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:29.332470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-446ba886-ad4f-48c4-81f3-21f5e2da0de6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:29.339999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-446ba886-ad4f-48c4-81f3-21f5e2da0de6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079799 Jun 10 13:21:29.340549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 852/1623] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:29 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:29.779624 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f3a0be80-6d35-4934-b99e-0935ff5e0c0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:29.780232 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f3a0be80-6d35-4934-b99e-0935ff5e0c0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:29.786853 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3a0be80-6d35-4934-b99e-0935ff5e0c0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:29.787306 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3a0be80-6d35-4934-b99e-0935ff5e0c0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:29.788057 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3a0be80-6d35-4934-b99e-0935ff5e0c0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:29.788486 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f3a0be80-6d35-4934-b99e-0935ff5e0c0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:29.911813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f3a0be80-6d35-4934-b99e-0935ff5e0c0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:29.919700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f3a0be80-6d35-4934-b99e-0935ff5e0c0b tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.143920 Jun 10 13:21:29.920261 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 772/1624] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:29 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:30.357200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-95a9f047-64ef-4347-aecb-0e9691445cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:30.357809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-95a9f047-64ef-4347-aecb-0e9691445cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:30.365095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95a9f047-64ef-4347-aecb-0e9691445cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:30.365526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95a9f047-64ef-4347-aecb-0e9691445cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:30.366281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95a9f047-64ef-4347-aecb-0e9691445cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:30.366712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95a9f047-64ef-4347-aecb-0e9691445cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:30.425130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-95a9f047-64ef-4347-aecb-0e9691445cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:30.432352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-95a9f047-64ef-4347-aecb-0e9691445cf2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.078987 Jun 10 13:21:30.432941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 853/1625] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:30 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:30.937180 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-12d6659e-aaa7-4509-a7c9-bf1c1b6ddf29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:30.937784 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-12d6659e-aaa7-4509-a7c9-bf1c1b6ddf29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:30.944405 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12d6659e-aaa7-4509-a7c9-bf1c1b6ddf29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:30.944893 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12d6659e-aaa7-4509-a7c9-bf1c1b6ddf29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:30.945646 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12d6659e-aaa7-4509-a7c9-bf1c1b6ddf29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:30.946075 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12d6659e-aaa7-4509-a7c9-bf1c1b6ddf29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:31.071045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-12d6659e-aaa7-4509-a7c9-bf1c1b6ddf29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:31.078794 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-12d6659e-aaa7-4509-a7c9-bf1c1b6ddf29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145414 Jun 10 13:21:31.079347 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 773/1626] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:30 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:31.449404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-50595ddb-0858-4ba0-a259-473e95e6ffdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:31.450019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-50595ddb-0858-4ba0-a259-473e95e6ffdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:31.455997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-50595ddb-0858-4ba0-a259-473e95e6ffdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:31.456438 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-50595ddb-0858-4ba0-a259-473e95e6ffdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:31.457231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-50595ddb-0858-4ba0-a259-473e95e6ffdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:31.457664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-50595ddb-0858-4ba0-a259-473e95e6ffdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:31.515896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-50595ddb-0858-4ba0-a259-473e95e6ffdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:31.523278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-50595ddb-0858-4ba0-a259-473e95e6ffdd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.077948 Jun 10 13:21:31.523832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 854/1627] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:31 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:32.095588 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3befd471-1745-485e-8a2c-0ce473b39b99 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:32.096103 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3befd471-1745-485e-8a2c-0ce473b39b99 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:32.102773 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3befd471-1745-485e-8a2c-0ce473b39b99 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:32.103212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3befd471-1745-485e-8a2c-0ce473b39b99 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:32.103958 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3befd471-1745-485e-8a2c-0ce473b39b99 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:32.104386 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3befd471-1745-485e-8a2c-0ce473b39b99 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:32.231902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3befd471-1745-485e-8a2c-0ce473b39b99 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:32.240170 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3befd471-1745-485e-8a2c-0ce473b39b99 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148474 Jun 10 13:21:32.240729 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 774/1628] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:32 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:32.540187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0df91caf-c2b6-4b91-a8d0-951799932e2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:32.540812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0df91caf-c2b6-4b91-a8d0-951799932e2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:32.547235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0df91caf-c2b6-4b91-a8d0-951799932e2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:32.547683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0df91caf-c2b6-4b91-a8d0-951799932e2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:32.549056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0df91caf-c2b6-4b91-a8d0-951799932e2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:32.549490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0df91caf-c2b6-4b91-a8d0-951799932e2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:32.553445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:21:32.610654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0df91caf-c2b6-4b91-a8d0-951799932e2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:32.618690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0df91caf-c2b6-4b91-a8d0-951799932e2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082392 Jun 10 13:21:32.619241 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 855/1629] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:32 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:33.257548 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ecccdf82-497d-4321-89a3-8de0d1a600e1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:33.258181 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ecccdf82-497d-4321-89a3-8de0d1a600e1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:33.264617 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ecccdf82-497d-4321-89a3-8de0d1a600e1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:33.265029 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ecccdf82-497d-4321-89a3-8de0d1a600e1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:33.265782 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ecccdf82-497d-4321-89a3-8de0d1a600e1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:33.266210 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ecccdf82-497d-4321-89a3-8de0d1a600e1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:33.390405 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ecccdf82-497d-4321-89a3-8de0d1a600e1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:33.398818 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ecccdf82-497d-4321-89a3-8de0d1a600e1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145136 Jun 10 13:21:33.399404 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 775/1630] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:33 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:33.635468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f35fb223-b104-41f0-b1a5-26d9c74ea906 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:33.635998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f35fb223-b104-41f0-b1a5-26d9c74ea906 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:33.642901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f35fb223-b104-41f0-b1a5-26d9c74ea906 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:33.643343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f35fb223-b104-41f0-b1a5-26d9c74ea906 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:33.644090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f35fb223-b104-41f0-b1a5-26d9c74ea906 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:33.644519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f35fb223-b104-41f0-b1a5-26d9c74ea906 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:33.702766 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f35fb223-b104-41f0-b1a5-26d9c74ea906 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:33.710657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f35fb223-b104-41f0-b1a5-26d9c74ea906 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079055 Jun 10 13:21:33.711213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 856/1631] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:33 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:34.416290 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f76154d3-d3f2-45b4-af8c-c7294d55b097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:34.416950 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f76154d3-d3f2-45b4-af8c-c7294d55b097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:34.423460 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f76154d3-d3f2-45b4-af8c-c7294d55b097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:34.423902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f76154d3-d3f2-45b4-af8c-c7294d55b097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:34.424650 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f76154d3-d3f2-45b4-af8c-c7294d55b097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:34.425117 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f76154d3-d3f2-45b4-af8c-c7294d55b097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:34.553638 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f76154d3-d3f2-45b4-af8c-c7294d55b097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:34.561460 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f76154d3-d3f2-45b4-af8c-c7294d55b097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148968 Jun 10 13:21:34.562026 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 776/1632] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:34 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:34.727826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f300f352-6709-4c83-949b-5abac3a7ed83 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:34.728512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f300f352-6709-4c83-949b-5abac3a7ed83 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:34.735633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f300f352-6709-4c83-949b-5abac3a7ed83 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:34.736518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f300f352-6709-4c83-949b-5abac3a7ed83 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:34.737377 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f300f352-6709-4c83-949b-5abac3a7ed83 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:34.737812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f300f352-6709-4c83-949b-5abac3a7ed83 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:34.799801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f300f352-6709-4c83-949b-5abac3a7ed83 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:34.807217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f300f352-6709-4c83-949b-5abac3a7ed83 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.084204 Jun 10 13:21:34.807773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 857/1633] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:34 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:35.578844 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e44b4ddb-93db-48fc-809f-a8c6ca84339c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:35.579487 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e44b4ddb-93db-48fc-809f-a8c6ca84339c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:35.586081 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e44b4ddb-93db-48fc-809f-a8c6ca84339c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:35.586477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e44b4ddb-93db-48fc-809f-a8c6ca84339c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:35.587232 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e44b4ddb-93db-48fc-809f-a8c6ca84339c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:35.587662 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e44b4ddb-93db-48fc-809f-a8c6ca84339c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:35.713254 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e44b4ddb-93db-48fc-809f-a8c6ca84339c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:35.721728 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e44b4ddb-93db-48fc-809f-a8c6ca84339c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146660 Jun 10 13:21:35.722300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 777/1634] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:35 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:35.824378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0cff540e-c11d-4346-932b-8ffd1955b810 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:35.825034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0cff540e-c11d-4346-932b-8ffd1955b810 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:35.831639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0cff540e-c11d-4346-932b-8ffd1955b810 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:35.832088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0cff540e-c11d-4346-932b-8ffd1955b810 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:35.832883 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0cff540e-c11d-4346-932b-8ffd1955b810 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:35.833323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0cff540e-c11d-4346-932b-8ffd1955b810 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:35.892752 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0cff540e-c11d-4346-932b-8ffd1955b810 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:35.900279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0cff540e-c11d-4346-932b-8ffd1955b810 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079756 Jun 10 13:21:35.900870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 858/1635] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:35 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:36.739007 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-587d98f6-6b6c-4411-8b25-5b439715de4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:36.739617 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-587d98f6-6b6c-4411-8b25-5b439715de4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:36.746140 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-587d98f6-6b6c-4411-8b25-5b439715de4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:36.746584 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-587d98f6-6b6c-4411-8b25-5b439715de4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:36.747335 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-587d98f6-6b6c-4411-8b25-5b439715de4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:36.747768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-587d98f6-6b6c-4411-8b25-5b439715de4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:36.765964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:21:36.877124 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-587d98f6-6b6c-4411-8b25-5b439715de4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:36.884999 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-587d98f6-6b6c-4411-8b25-5b439715de4c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.149818 Jun 10 13:21:36.885566 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 778/1636] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:36 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:36.916878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-58738228-b029-4955-9d00-748f14d9edba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:36.917539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-58738228-b029-4955-9d00-748f14d9edba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:36.924579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58738228-b029-4955-9d00-748f14d9edba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:36.925047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58738228-b029-4955-9d00-748f14d9edba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:36.925800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58738228-b029-4955-9d00-748f14d9edba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:36.926232 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58738228-b029-4955-9d00-748f14d9edba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:36.988004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-58738228-b029-4955-9d00-748f14d9edba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:36.995443 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-58738228-b029-4955-9d00-748f14d9edba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082413 Jun 10 13:21:36.995938 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 859/1637] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:36 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:37.902693 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-987ae8f7-46df-4dd1-921b-eae59f27429f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:37.903297 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-987ae8f7-46df-4dd1-921b-eae59f27429f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:37.909744 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-987ae8f7-46df-4dd1-921b-eae59f27429f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:37.910191 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-987ae8f7-46df-4dd1-921b-eae59f27429f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:37.911012 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-987ae8f7-46df-4dd1-921b-eae59f27429f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:37.911438 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-987ae8f7-46df-4dd1-921b-eae59f27429f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:38.011475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2f94a552-ef1d-4928-aab0-a0ca0d0c05d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:38.012094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2f94a552-ef1d-4928-aab0-a0ca0d0c05d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:38.018552 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f94a552-ef1d-4928-aab0-a0ca0d0c05d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:38.019009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f94a552-ef1d-4928-aab0-a0ca0d0c05d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:38.019777 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f94a552-ef1d-4928-aab0-a0ca0d0c05d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:38.020213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f94a552-ef1d-4928-aab0-a0ca0d0c05d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:38.036246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-987ae8f7-46df-4dd1-921b-eae59f27429f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:38.043998 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-987ae8f7-46df-4dd1-921b-eae59f27429f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145098 Jun 10 13:21:38.044561 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 779/1638] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:37 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:38.078968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2f94a552-ef1d-4928-aab0-a0ca0d0c05d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:38.086405 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2f94a552-ef1d-4928-aab0-a0ca0d0c05d7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.078841 Jun 10 13:21:38.086978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 860/1639] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:38 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:39.061761 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f1dd9e6c-9dbd-4c43-a48a-dfc0da2c7160 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:39.062367 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f1dd9e6c-9dbd-4c43-a48a-dfc0da2c7160 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:39.068856 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f1dd9e6c-9dbd-4c43-a48a-dfc0da2c7160 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:39.069307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f1dd9e6c-9dbd-4c43-a48a-dfc0da2c7160 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:39.070082 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f1dd9e6c-9dbd-4c43-a48a-dfc0da2c7160 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:39.070499 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f1dd9e6c-9dbd-4c43-a48a-dfc0da2c7160 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:39.103358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-99069a57-bb75-4f2e-bb27-cf53db05450f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:39.104098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-99069a57-bb75-4f2e-bb27-cf53db05450f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:39.111222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99069a57-bb75-4f2e-bb27-cf53db05450f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:39.111674 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99069a57-bb75-4f2e-bb27-cf53db05450f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:39.112451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99069a57-bb75-4f2e-bb27-cf53db05450f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:39.112928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99069a57-bb75-4f2e-bb27-cf53db05450f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:39.172543 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-99069a57-bb75-4f2e-bb27-cf53db05450f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:39.180894 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-99069a57-bb75-4f2e-bb27-cf53db05450f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.081616 Jun 10 13:21:39.181485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 861/1640] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:39 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:39.196908 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f1dd9e6c-9dbd-4c43-a48a-dfc0da2c7160 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:39.204846 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f1dd9e6c-9dbd-4c43-a48a-dfc0da2c7160 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146942 Jun 10 13:21:39.205408 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 780/1641] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:39 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:40.196673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5ed80dbe-f95e-48dd-9674-0d53d2f9fcf6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:40.197302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5ed80dbe-f95e-48dd-9674-0d53d2f9fcf6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:40.203872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5ed80dbe-f95e-48dd-9674-0d53d2f9fcf6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:40.204314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5ed80dbe-f95e-48dd-9674-0d53d2f9fcf6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:40.205095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5ed80dbe-f95e-48dd-9674-0d53d2f9fcf6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:40.205525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5ed80dbe-f95e-48dd-9674-0d53d2f9fcf6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:40.221753 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-554344d1-b3ea-4b57-b0b6-3cfdee821d69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:40.222512 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-554344d1-b3ea-4b57-b0b6-3cfdee821d69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:40.229117 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-554344d1-b3ea-4b57-b0b6-3cfdee821d69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:40.229560 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-554344d1-b3ea-4b57-b0b6-3cfdee821d69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:40.230313 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-554344d1-b3ea-4b57-b0b6-3cfdee821d69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:40.230743 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-554344d1-b3ea-4b57-b0b6-3cfdee821d69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:40.269177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5ed80dbe-f95e-48dd-9674-0d53d2f9fcf6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:40.276984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5ed80dbe-f95e-48dd-9674-0d53d2f9fcf6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.084022 Jun 10 13:21:40.277571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 862/1642] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:40 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:40.365801 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-554344d1-b3ea-4b57-b0b6-3cfdee821d69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:40.373541 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-554344d1-b3ea-4b57-b0b6-3cfdee821d69 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.155715 Jun 10 13:21:40.374095 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 781/1643] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:40 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:41.295005 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5c4ec539-c967-46c0-a40c-03af9a2f7be8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:41.295695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5c4ec539-c967-46c0-a40c-03af9a2f7be8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:41.302887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c4ec539-c967-46c0-a40c-03af9a2f7be8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:41.303325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c4ec539-c967-46c0-a40c-03af9a2f7be8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:41.304081 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c4ec539-c967-46c0-a40c-03af9a2f7be8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:41.304508 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c4ec539-c967-46c0-a40c-03af9a2f7be8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:41.369508 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5c4ec539-c967-46c0-a40c-03af9a2f7be8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:41.377090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5c4ec539-c967-46c0-a40c-03af9a2f7be8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.085881 Jun 10 13:21:41.377653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 863/1644] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:41 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:41.390404 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b21db95c-cc4e-42d6-bcfa-2fc585e3555f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:41.391009 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b21db95c-cc4e-42d6-bcfa-2fc585e3555f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:41.397434 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b21db95c-cc4e-42d6-bcfa-2fc585e3555f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:41.397868 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b21db95c-cc4e-42d6-bcfa-2fc585e3555f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:41.398623 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b21db95c-cc4e-42d6-bcfa-2fc585e3555f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:41.399056 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b21db95c-cc4e-42d6-bcfa-2fc585e3555f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:41.523413 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b21db95c-cc4e-42d6-bcfa-2fc585e3555f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:41.531255 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b21db95c-cc4e-42d6-bcfa-2fc585e3555f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144631 Jun 10 13:21:41.531814 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 782/1645] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:41 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:42.393844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c528dd3b-2a0c-4a98-bafc-2bd9c108a8b9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:42.394462 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c528dd3b-2a0c-4a98-bafc-2bd9c108a8b9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:42.401033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c528dd3b-2a0c-4a98-bafc-2bd9c108a8b9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:42.401486 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c528dd3b-2a0c-4a98-bafc-2bd9c108a8b9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:42.402248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c528dd3b-2a0c-4a98-bafc-2bd9c108a8b9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:42.402685 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c528dd3b-2a0c-4a98-bafc-2bd9c108a8b9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:42.461159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c528dd3b-2a0c-4a98-bafc-2bd9c108a8b9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:42.468496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c528dd3b-2a0c-4a98-bafc-2bd9c108a8b9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.078505 Jun 10 13:21:42.469090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 864/1646] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:42 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:42.549272 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-288bfe24-1ed4-476e-9d46-7760942d95d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:42.549810 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-288bfe24-1ed4-476e-9d46-7760942d95d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:42.556116 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-288bfe24-1ed4-476e-9d46-7760942d95d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:42.556560 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-288bfe24-1ed4-476e-9d46-7760942d95d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:42.557334 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-288bfe24-1ed4-476e-9d46-7760942d95d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:42.557762 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-288bfe24-1ed4-476e-9d46-7760942d95d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:42.683717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-288bfe24-1ed4-476e-9d46-7760942d95d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:42.691450 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-288bfe24-1ed4-476e-9d46-7760942d95d7 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146001 Jun 10 13:21:42.692004 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 783/1647] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:42 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:43.487309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-33b3e46b-c9e8-4fbe-9eb4-d28bfbd40ad3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:43.487911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-33b3e46b-c9e8-4fbe-9eb4-d28bfbd40ad3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:43.490206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=53 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:21:43.494876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33b3e46b-c9e8-4fbe-9eb4-d28bfbd40ad3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:43.495318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33b3e46b-c9e8-4fbe-9eb4-d28bfbd40ad3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:43.496096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33b3e46b-c9e8-4fbe-9eb4-d28bfbd40ad3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:43.496530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33b3e46b-c9e8-4fbe-9eb4-d28bfbd40ad3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:43.557888 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-33b3e46b-c9e8-4fbe-9eb4-d28bfbd40ad3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:43.565121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-33b3e46b-c9e8-4fbe-9eb4-d28bfbd40ad3 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082262 Jun 10 13:21:43.565683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 865/1648] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:43 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:43.708270 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b73951b9-62a0-4b84-9869-93414b250f30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:43.708978 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b73951b9-62a0-4b84-9869-93414b250f30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:43.715435 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b73951b9-62a0-4b84-9869-93414b250f30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:43.715881 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b73951b9-62a0-4b84-9869-93414b250f30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:43.716641 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b73951b9-62a0-4b84-9869-93414b250f30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:43.717107 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b73951b9-62a0-4b84-9869-93414b250f30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:43.841713 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b73951b9-62a0-4b84-9869-93414b250f30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:43.850374 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b73951b9-62a0-4b84-9869-93414b250f30 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145948 Jun 10 13:21:43.850937 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 784/1649] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:43 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:44.582587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-051e30c1-7a83-4ef8-af8c-511a26990c91 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:44.583129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-051e30c1-7a83-4ef8-af8c-511a26990c91 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:44.589569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-051e30c1-7a83-4ef8-af8c-511a26990c91 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:44.590015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-051e30c1-7a83-4ef8-af8c-511a26990c91 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:44.590775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-051e30c1-7a83-4ef8-af8c-511a26990c91 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:44.591205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-051e30c1-7a83-4ef8-af8c-511a26990c91 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:44.651054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-051e30c1-7a83-4ef8-af8c-511a26990c91 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:44.658543 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-051e30c1-7a83-4ef8-af8c-511a26990c91 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080526 Jun 10 13:21:44.659103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 866/1650] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:44 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:44.868454 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c43121d0-8837-4300-bd5c-02cc4b8ae77c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:44.869007 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c43121d0-8837-4300-bd5c-02cc4b8ae77c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:44.875403 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c43121d0-8837-4300-bd5c-02cc4b8ae77c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:44.876456 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c43121d0-8837-4300-bd5c-02cc4b8ae77c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:44.877232 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c43121d0-8837-4300-bd5c-02cc4b8ae77c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:44.877661 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c43121d0-8837-4300-bd5c-02cc4b8ae77c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:45.002430 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c43121d0-8837-4300-bd5c-02cc4b8ae77c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:45.010244 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c43121d0-8837-4300-bd5c-02cc4b8ae77c tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145711 Jun 10 13:21:45.010800 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 785/1651] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:44 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:45.319695 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:21:45.319900 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Fri Jun 10 13:21:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:21:45.675128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-125a5c5a-2b46-4a8c-ae41-56a58fef7647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:45.675666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-125a5c5a-2b46-4a8c-ae41-56a58fef7647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:45.682172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-125a5c5a-2b46-4a8c-ae41-56a58fef7647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:45.682619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-125a5c5a-2b46-4a8c-ae41-56a58fef7647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:45.683374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-125a5c5a-2b46-4a8c-ae41-56a58fef7647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:45.683807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-125a5c5a-2b46-4a8c-ae41-56a58fef7647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:45.745586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-125a5c5a-2b46-4a8c-ae41-56a58fef7647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:45.753609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-125a5c5a-2b46-4a8c-ae41-56a58fef7647 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082408 Jun 10 13:21:45.757230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 867/1652] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:45 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:46.027504 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-14572f24-c9c9-4547-899b-bdb9a5b5dbb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:46.028108 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-14572f24-c9c9-4547-899b-bdb9a5b5dbb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:46.034853 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14572f24-c9c9-4547-899b-bdb9a5b5dbb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:46.035296 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14572f24-c9c9-4547-899b-bdb9a5b5dbb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:46.036045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14572f24-c9c9-4547-899b-bdb9a5b5dbb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:46.036469 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-14572f24-c9c9-4547-899b-bdb9a5b5dbb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:46.162431 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-14572f24-c9c9-4547-899b-bdb9a5b5dbb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:46.170306 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-14572f24-c9c9-4547-899b-bdb9a5b5dbb1 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146972 Jun 10 13:21:46.170867 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 786/1653] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:46 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:46.770995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8b51ae44-2f8d-4b16-99ec-95c609fd0b90 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:46.771607 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8b51ae44-2f8d-4b16-99ec-95c609fd0b90 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:46.778065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8b51ae44-2f8d-4b16-99ec-95c609fd0b90 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:46.778505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8b51ae44-2f8d-4b16-99ec-95c609fd0b90 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:46.779261 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8b51ae44-2f8d-4b16-99ec-95c609fd0b90 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:46.779685 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8b51ae44-2f8d-4b16-99ec-95c609fd0b90 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:46.839408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8b51ae44-2f8d-4b16-99ec-95c609fd0b90 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:46.846969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8b51ae44-2f8d-4b16-99ec-95c609fd0b90 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080535 Jun 10 13:21:46.847532 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 868/1654] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:46 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:47.187595 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6198e257-251d-404c-a629-5db5a164a557 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:47.188867 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6198e257-251d-404c-a629-5db5a164a557 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:47.191140 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=51 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:21:47.195813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6198e257-251d-404c-a629-5db5a164a557 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:47.196250 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6198e257-251d-404c-a629-5db5a164a557 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:47.197252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6198e257-251d-404c-a629-5db5a164a557 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:47.197682 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6198e257-251d-404c-a629-5db5a164a557 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:47.323382 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6198e257-251d-404c-a629-5db5a164a557 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:47.331295 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6198e257-251d-404c-a629-5db5a164a557 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147548 Jun 10 13:21:47.331854 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 787/1655] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:47 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:47.864906 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0bb3db36-b3a1-4f7f-8f1b-75df76ba333a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:47.865582 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0bb3db36-b3a1-4f7f-8f1b-75df76ba333a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:47.871926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0bb3db36-b3a1-4f7f-8f1b-75df76ba333a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:47.872375 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0bb3db36-b3a1-4f7f-8f1b-75df76ba333a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:47.873155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0bb3db36-b3a1-4f7f-8f1b-75df76ba333a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:47.873587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0bb3db36-b3a1-4f7f-8f1b-75df76ba333a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:47.932153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0bb3db36-b3a1-4f7f-8f1b-75df76ba333a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:47.940188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0bb3db36-b3a1-4f7f-8f1b-75df76ba333a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079237 Jun 10 13:21:47.940746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 869/1656] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:47 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:48.347829 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-889e767a-ebd4-40ed-9da7-a937b2b96f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:48.348434 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-889e767a-ebd4-40ed-9da7-a937b2b96f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:48.355235 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-889e767a-ebd4-40ed-9da7-a937b2b96f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:48.355681 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-889e767a-ebd4-40ed-9da7-a937b2b96f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:48.356441 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-889e767a-ebd4-40ed-9da7-a937b2b96f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:48.356909 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-889e767a-ebd4-40ed-9da7-a937b2b96f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:48.484346 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-889e767a-ebd4-40ed-9da7-a937b2b96f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:48.492108 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-889e767a-ebd4-40ed-9da7-a937b2b96f5d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148653 Jun 10 13:21:48.492664 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 788/1657] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:48 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:48.956968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-33ce23cb-b363-459d-b8f6-f5f9ac6f618a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:48.957573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-33ce23cb-b363-459d-b8f6-f5f9ac6f618a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:48.964070 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33ce23cb-b363-459d-b8f6-f5f9ac6f618a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:48.964509 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33ce23cb-b363-459d-b8f6-f5f9ac6f618a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:48.965297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33ce23cb-b363-459d-b8f6-f5f9ac6f618a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:48.965726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33ce23cb-b363-459d-b8f6-f5f9ac6f618a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:49.028414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-33ce23cb-b363-459d-b8f6-f5f9ac6f618a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:49.036432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-33ce23cb-b363-459d-b8f6-f5f9ac6f618a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.083344 Jun 10 13:21:49.037017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 870/1658] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:48 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:49.508271 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e205ab9d-caac-4f7c-9fd0-5bbc2e8fc942 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:49.508902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e205ab9d-caac-4f7c-9fd0-5bbc2e8fc942 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:49.515484 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e205ab9d-caac-4f7c-9fd0-5bbc2e8fc942 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:49.515929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e205ab9d-caac-4f7c-9fd0-5bbc2e8fc942 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:49.517341 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e205ab9d-caac-4f7c-9fd0-5bbc2e8fc942 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:49.517771 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e205ab9d-caac-4f7c-9fd0-5bbc2e8fc942 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:49.649605 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e205ab9d-caac-4f7c-9fd0-5bbc2e8fc942 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:49.657456 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e205ab9d-caac-4f7c-9fd0-5bbc2e8fc942 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.152834 Jun 10 13:21:49.658016 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 789/1659] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:49 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:50.053364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-364dee1f-0301-4b06-b775-fae0a6abef42 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:50.053965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-364dee1f-0301-4b06-b775-fae0a6abef42 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:50.060304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-364dee1f-0301-4b06-b775-fae0a6abef42 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:50.060744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-364dee1f-0301-4b06-b775-fae0a6abef42 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:50.061663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-364dee1f-0301-4b06-b775-fae0a6abef42 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:50.062093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-364dee1f-0301-4b06-b775-fae0a6abef42 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:50.124738 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-364dee1f-0301-4b06-b775-fae0a6abef42 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:50.128850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:21:50.133489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-364dee1f-0301-4b06-b775-fae0a6abef42 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.083972 Jun 10 13:21:50.134156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 871/1660] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:50 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:50.674440 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b7aa3e49-9d81-4a9b-8ae6-8371facae0ea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:50.674979 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b7aa3e49-9d81-4a9b-8ae6-8371facae0ea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:50.681489 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b7aa3e49-9d81-4a9b-8ae6-8371facae0ea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:50.681940 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b7aa3e49-9d81-4a9b-8ae6-8371facae0ea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:50.682697 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b7aa3e49-9d81-4a9b-8ae6-8371facae0ea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:50.683134 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b7aa3e49-9d81-4a9b-8ae6-8371facae0ea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:50.809831 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b7aa3e49-9d81-4a9b-8ae6-8371facae0ea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:50.817747 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b7aa3e49-9d81-4a9b-8ae6-8371facae0ea tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147220 Jun 10 13:21:50.818314 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 790/1661] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:50 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:51.151089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8dc230de-f477-451c-92bf-314a6279bf7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:51.151690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8dc230de-f477-451c-92bf-314a6279bf7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:51.158460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8dc230de-f477-451c-92bf-314a6279bf7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:51.158902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8dc230de-f477-451c-92bf-314a6279bf7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:51.159654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8dc230de-f477-451c-92bf-314a6279bf7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:51.160078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8dc230de-f477-451c-92bf-314a6279bf7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:51.220122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8dc230de-f477-451c-92bf-314a6279bf7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:51.227849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8dc230de-f477-451c-92bf-314a6279bf7b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.081135 Jun 10 13:21:51.228406 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 872/1662] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:51 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:51.834757 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b1d89428-e2fd-4afd-a649-8074e81d81c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:51.835366 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b1d89428-e2fd-4afd-a649-8074e81d81c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:51.842112 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b1d89428-e2fd-4afd-a649-8074e81d81c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:51.842599 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b1d89428-e2fd-4afd-a649-8074e81d81c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:51.843339 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b1d89428-e2fd-4afd-a649-8074e81d81c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:51.843764 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b1d89428-e2fd-4afd-a649-8074e81d81c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:51.969976 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b1d89428-e2fd-4afd-a649-8074e81d81c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:51.978051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b1d89428-e2fd-4afd-a649-8074e81d81c0 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147229 Jun 10 13:21:51.978600 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 791/1663] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:51 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:52.244929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d1698540-286b-4f45-aba9-7ff35990cedd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:52.245595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d1698540-286b-4f45-aba9-7ff35990cedd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:52.251961 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d1698540-286b-4f45-aba9-7ff35990cedd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:52.252406 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d1698540-286b-4f45-aba9-7ff35990cedd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:52.253197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d1698540-286b-4f45-aba9-7ff35990cedd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:52.254788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d1698540-286b-4f45-aba9-7ff35990cedd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:52.317905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d1698540-286b-4f45-aba9-7ff35990cedd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:52.325435 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d1698540-286b-4f45-aba9-7ff35990cedd tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.084424 Jun 10 13:21:52.326006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 873/1664] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:52 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:52.995502 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f9bab25a-4b72-44e7-9228-25fdb8b2fc29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:52.996108 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f9bab25a-4b72-44e7-9228-25fdb8b2fc29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:53.003314 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f9bab25a-4b72-44e7-9228-25fdb8b2fc29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:53.003766 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f9bab25a-4b72-44e7-9228-25fdb8b2fc29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:53.004531 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f9bab25a-4b72-44e7-9228-25fdb8b2fc29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:53.005001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f9bab25a-4b72-44e7-9228-25fdb8b2fc29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:53.133371 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f9bab25a-4b72-44e7-9228-25fdb8b2fc29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:53.141339 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f9bab25a-4b72-44e7-9228-25fdb8b2fc29 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.149650 Jun 10 13:21:53.141896 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 792/1665] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:52 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:53.343159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ea85674b-49c7-47b7-907f-de0c9ca14016 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:53.343769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ea85674b-49c7-47b7-907f-de0c9ca14016 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:53.350810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ea85674b-49c7-47b7-907f-de0c9ca14016 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:53.351250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ea85674b-49c7-47b7-907f-de0c9ca14016 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:53.352277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ea85674b-49c7-47b7-907f-de0c9ca14016 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:53.352707 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ea85674b-49c7-47b7-907f-de0c9ca14016 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:53.412326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ea85674b-49c7-47b7-907f-de0c9ca14016 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:53.419870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ea85674b-49c7-47b7-907f-de0c9ca14016 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080673 Jun 10 13:21:53.420433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 874/1666] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:53 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:54.158892 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8a6d5b5e-7b93-48a6-8cbd-10064415d9cf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:54.159527 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8a6d5b5e-7b93-48a6-8cbd-10064415d9cf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:54.165917 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8a6d5b5e-7b93-48a6-8cbd-10064415d9cf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:54.166354 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8a6d5b5e-7b93-48a6-8cbd-10064415d9cf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:54.167101 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8a6d5b5e-7b93-48a6-8cbd-10064415d9cf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:54.167547 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8a6d5b5e-7b93-48a6-8cbd-10064415d9cf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:54.295335 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8a6d5b5e-7b93-48a6-8cbd-10064415d9cf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:54.306254 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8a6d5b5e-7b93-48a6-8cbd-10064415d9cf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.151369 Jun 10 13:21:54.307242 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 793/1667] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:54 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:54.436572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f2acdcc5-5356-4838-922c-a2eac701414d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:54.437150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f2acdcc5-5356-4838-922c-a2eac701414d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:54.443674 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f2acdcc5-5356-4838-922c-a2eac701414d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:54.444124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f2acdcc5-5356-4838-922c-a2eac701414d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:54.444917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f2acdcc5-5356-4838-922c-a2eac701414d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:54.445352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f2acdcc5-5356-4838-922c-a2eac701414d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:54.507294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f2acdcc5-5356-4838-922c-a2eac701414d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:54.514701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f2acdcc5-5356-4838-922c-a2eac701414d tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082063 Jun 10 13:21:54.515256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 875/1668] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:54 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:55.324146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-61808ca5-14e0-4df7-b078-bee44e74ba26 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:55.324824 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-61808ca5-14e0-4df7-b078-bee44e74ba26 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:55.331635 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-61808ca5-14e0-4df7-b078-bee44e74ba26 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:55.332081 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-61808ca5-14e0-4df7-b078-bee44e74ba26 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:55.332883 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-61808ca5-14e0-4df7-b078-bee44e74ba26 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:55.333309 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-61808ca5-14e0-4df7-b078-bee44e74ba26 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:55.460208 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-61808ca5-14e0-4df7-b078-bee44e74ba26 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:55.468057 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-61808ca5-14e0-4df7-b078-bee44e74ba26 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148146 Jun 10 13:21:55.468625 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 794/1669] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:55 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:55.531273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-34050028-f7d1-43ed-90e8-fc3b873b008b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:55.531832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-34050028-f7d1-43ed-90e8-fc3b873b008b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:55.538915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-34050028-f7d1-43ed-90e8-fc3b873b008b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:55.539363 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-34050028-f7d1-43ed-90e8-fc3b873b008b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:55.540159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-34050028-f7d1-43ed-90e8-fc3b873b008b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:55.540591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-34050028-f7d1-43ed-90e8-fc3b873b008b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:55.601281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-34050028-f7d1-43ed-90e8-fc3b873b008b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:55.608780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-34050028-f7d1-43ed-90e8-fc3b873b008b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.081377 Jun 10 13:21:55.609339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 876/1670] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:55 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:56.485865 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d4b6fa1c-d88b-4b4e-b3ce-3350db05af65 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:56.486477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d4b6fa1c-d88b-4b4e-b3ce-3350db05af65 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:56.492978 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d4b6fa1c-d88b-4b4e-b3ce-3350db05af65 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:56.493427 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d4b6fa1c-d88b-4b4e-b3ce-3350db05af65 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:56.494182 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d4b6fa1c-d88b-4b4e-b3ce-3350db05af65 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:56.494616 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d4b6fa1c-d88b-4b4e-b3ce-3350db05af65 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:56.498445 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:21:56.627300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-676923c2-daac-41c5-9186-56655240b848 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:56.627846 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d4b6fa1c-d88b-4b4e-b3ce-3350db05af65 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:56.627968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-676923c2-daac-41c5-9186-56655240b848 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:56.634538 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-676923c2-daac-41c5-9186-56655240b848 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:56.634986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-676923c2-daac-41c5-9186-56655240b848 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:56.635756 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-676923c2-daac-41c5-9186-56655240b848 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:56.636189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-676923c2-daac-41c5-9186-56655240b848 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:56.638795 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d4b6fa1c-d88b-4b4e-b3ce-3350db05af65 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.156458 Jun 10 13:21:56.639447 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 795/1671] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:56 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:56.695416 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-676923c2-daac-41c5-9186-56655240b848 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:56.702763 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-676923c2-daac-41c5-9186-56655240b848 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079963 Jun 10 13:21:56.703322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 877/1672] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:56 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:57.655725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d4262025-fd0a-4609-874f-8a1494cef30e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:57.656253 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d4262025-fd0a-4609-874f-8a1494cef30e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:57.663301 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d4262025-fd0a-4609-874f-8a1494cef30e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:57.663741 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d4262025-fd0a-4609-874f-8a1494cef30e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:57.664483 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d4262025-fd0a-4609-874f-8a1494cef30e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:57.664936 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d4262025-fd0a-4609-874f-8a1494cef30e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:57.719254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c04d51b2-3951-4fd4-a50e-514bf4daede4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:57.719858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c04d51b2-3951-4fd4-a50e-514bf4daede4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:57.727557 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c04d51b2-3951-4fd4-a50e-514bf4daede4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:57.728013 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c04d51b2-3951-4fd4-a50e-514bf4daede4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:57.728822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c04d51b2-3951-4fd4-a50e-514bf4daede4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:57.729386 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c04d51b2-3951-4fd4-a50e-514bf4daede4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:57.790068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c04d51b2-3951-4fd4-a50e-514bf4daede4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:57.793595 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d4262025-fd0a-4609-874f-8a1494cef30e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:57.797602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c04d51b2-3951-4fd4-a50e-514bf4daede4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082112 Jun 10 13:21:57.798169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 878/1673] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:57 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:57.801363 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d4262025-fd0a-4609-874f-8a1494cef30e tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.149493 Jun 10 13:21:57.801906 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 796/1674] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:57 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:58.814508 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e7f15a42-b044-41d7-8457-2fe8c9316b6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:58.815114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e7f15a42-b044-41d7-8457-2fe8c9316b6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:58.817881 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-173ed3a2-172a-47a2-b708-8332d98ced01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:58.818513 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-173ed3a2-172a-47a2-b708-8332d98ced01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:58.821769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7f15a42-b044-41d7-8457-2fe8c9316b6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:58.822251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7f15a42-b044-41d7-8457-2fe8c9316b6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:58.823025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7f15a42-b044-41d7-8457-2fe8c9316b6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:58.823463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7f15a42-b044-41d7-8457-2fe8c9316b6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:58.824948 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-173ed3a2-172a-47a2-b708-8332d98ced01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:58.825398 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-173ed3a2-172a-47a2-b708-8332d98ced01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:58.826145 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-173ed3a2-172a-47a2-b708-8332d98ced01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:58.826604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-173ed3a2-172a-47a2-b708-8332d98ced01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:58.888091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e7f15a42-b044-41d7-8457-2fe8c9316b6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:58.896094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e7f15a42-b044-41d7-8457-2fe8c9316b6e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.085469 Jun 10 13:21:58.896647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 879/1675] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:58 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:21:58.952407 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-173ed3a2-172a-47a2-b708-8332d98ced01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:58.960817 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-173ed3a2-172a-47a2-b708-8332d98ced01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146784 Jun 10 13:21:58.961369 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 797/1676] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:58 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:21:59.913135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7da069b3-41ff-4cde-a705-74beb0df58eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:59.913742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7da069b3-41ff-4cde-a705-74beb0df58eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:59.920322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7da069b3-41ff-4cde-a705-74beb0df58eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:59.920778 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7da069b3-41ff-4cde-a705-74beb0df58eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:59.921536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7da069b3-41ff-4cde-a705-74beb0df58eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:59.921962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7da069b3-41ff-4cde-a705-74beb0df58eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:59.979151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2f7dab8d-628b-4f6b-ac96-b9520e5b62c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:21:59.979845 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2f7dab8d-628b-4f6b-ac96-b9520e5b62c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:21:59.982595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7da069b3-41ff-4cde-a705-74beb0df58eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:21:59.987477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2f7dab8d-628b-4f6b-ac96-b9520e5b62c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:59.987958 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2f7dab8d-628b-4f6b-ac96-b9520e5b62c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:59.988943 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2f7dab8d-628b-4f6b-ac96-b9520e5b62c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:21:59.989410 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2f7dab8d-628b-4f6b-ac96-b9520e5b62c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:21:59.991109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7da069b3-41ff-4cde-a705-74beb0df58eb tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.081889 Jun 10 13:21:59.991709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 880/1677] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:59 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:00.121161 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2f7dab8d-628b-4f6b-ac96-b9520e5b62c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:00.129031 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2f7dab8d-628b-4f6b-ac96-b9520e5b62c5 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.153892 Jun 10 13:22:00.129590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 798/1678] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:21:59 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:22:01.008818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-76781da5-b137-4f66-b9d8-f4ff94ec3ce6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:01.009432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-76781da5-b137-4f66-b9d8-f4ff94ec3ce6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:01.015581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76781da5-b137-4f66-b9d8-f4ff94ec3ce6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:01.016021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76781da5-b137-4f66-b9d8-f4ff94ec3ce6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:01.016794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76781da5-b137-4f66-b9d8-f4ff94ec3ce6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:01.017249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76781da5-b137-4f66-b9d8-f4ff94ec3ce6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:01.075932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-76781da5-b137-4f66-b9d8-f4ff94ec3ce6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:01.083668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-76781da5-b137-4f66-b9d8-f4ff94ec3ce6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079893 Jun 10 13:22:01.084220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 881/1679] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:01 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:01.146115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7f60efc2-2d59-48ec-9ab0-ed3eae81e36a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:01.146837 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7f60efc2-2d59-48ec-9ab0-ed3eae81e36a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:01.153228 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7f60efc2-2d59-48ec-9ab0-ed3eae81e36a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:01.153668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7f60efc2-2d59-48ec-9ab0-ed3eae81e36a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:01.155095 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7f60efc2-2d59-48ec-9ab0-ed3eae81e36a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:01.155544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7f60efc2-2d59-48ec-9ab0-ed3eae81e36a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:01.280469 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7f60efc2-2d59-48ec-9ab0-ed3eae81e36a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:01.288970 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7f60efc2-2d59-48ec-9ab0-ed3eae81e36a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.147102 Jun 10 13:22:01.289539 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 799/1680] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:01 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:02.102018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e2109138-a3ce-4ea2-ac9e-a68be2d1e280 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:02.102540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e2109138-a3ce-4ea2-ac9e-a68be2d1e280 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:02.109203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e2109138-a3ce-4ea2-ac9e-a68be2d1e280 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:02.109654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e2109138-a3ce-4ea2-ac9e-a68be2d1e280 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:02.110414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e2109138-a3ce-4ea2-ac9e-a68be2d1e280 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:02.110842 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e2109138-a3ce-4ea2-ac9e-a68be2d1e280 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:02.169578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e2109138-a3ce-4ea2-ac9e-a68be2d1e280 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:02.176925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e2109138-a3ce-4ea2-ac9e-a68be2d1e280 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079069 Jun 10 13:22:02.177480 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 882/1681] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:02 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:02.305762 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6a5550ef-6e0b-471c-a8bd-256feb2a06d4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:02.306399 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6a5550ef-6e0b-471c-a8bd-256feb2a06d4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:02.312935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6a5550ef-6e0b-471c-a8bd-256feb2a06d4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:02.313382 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6a5550ef-6e0b-471c-a8bd-256feb2a06d4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:02.314134 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6a5550ef-6e0b-471c-a8bd-256feb2a06d4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:02.314563 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6a5550ef-6e0b-471c-a8bd-256feb2a06d4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:02.449109 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6a5550ef-6e0b-471c-a8bd-256feb2a06d4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:02.457590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6a5550ef-6e0b-471c-a8bd-256feb2a06d4 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.155622 Jun 10 13:22:02.458154 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 800/1682] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:02 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:03.193608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7b6af294-f2d3-4829-b4c2-cbacf63bb56f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:03.194246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7b6af294-f2d3-4829-b4c2-cbacf63bb56f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:03.200955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b6af294-f2d3-4829-b4c2-cbacf63bb56f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:03.201432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b6af294-f2d3-4829-b4c2-cbacf63bb56f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:03.202186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b6af294-f2d3-4829-b4c2-cbacf63bb56f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:03.202616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b6af294-f2d3-4829-b4c2-cbacf63bb56f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:03.268107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7b6af294-f2d3-4829-b4c2-cbacf63bb56f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:03.275428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7b6af294-f2d3-4829-b4c2-cbacf63bb56f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.085741 Jun 10 13:22:03.275989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 883/1683] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:03 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:03.475492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-e9d876bf-040a-4ac4-8458-09665bbb6dc8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:03.476034 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-e9d876bf-040a-4ac4-8458-09665bbb6dc8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:03.482654 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9d876bf-040a-4ac4-8458-09665bbb6dc8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:03.483096 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9d876bf-040a-4ac4-8458-09665bbb6dc8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:03.483839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9d876bf-040a-4ac4-8458-09665bbb6dc8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:03.484264 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-e9d876bf-040a-4ac4-8458-09665bbb6dc8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:03.609818 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-e9d876bf-040a-4ac4-8458-09665bbb6dc8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:03.617590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-e9d876bf-040a-4ac4-8458-09665bbb6dc8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146625 Jun 10 13:22:03.618150 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 801/1684] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:03 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:04.294345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-39dfe5cd-c180-481c-bc84-1c77061862e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:04.294950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-39dfe5cd-c180-481c-bc84-1c77061862e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:04.301407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39dfe5cd-c180-481c-bc84-1c77061862e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:04.301850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39dfe5cd-c180-481c-bc84-1c77061862e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:04.302594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39dfe5cd-c180-481c-bc84-1c77061862e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:04.303023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39dfe5cd-c180-481c-bc84-1c77061862e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:04.364078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-39dfe5cd-c180-481c-bc84-1c77061862e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:04.371539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-39dfe5cd-c180-481c-bc84-1c77061862e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082340 Jun 10 13:22:04.372092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 884/1685] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:04 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:04.635048 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-28057cff-d7eb-42dc-bacd-a998550acc85 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:04.635655 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-28057cff-d7eb-42dc-bacd-a998550acc85 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:04.642125 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-28057cff-d7eb-42dc-bacd-a998550acc85 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:04.642570 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-28057cff-d7eb-42dc-bacd-a998550acc85 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:04.643349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-28057cff-d7eb-42dc-bacd-a998550acc85 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:04.644438 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-28057cff-d7eb-42dc-bacd-a998550acc85 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:04.773435 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-28057cff-d7eb-42dc-bacd-a998550acc85 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:04.781441 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-28057cff-d7eb-42dc-bacd-a998550acc85 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.150243 Jun 10 13:22:04.782000 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 802/1686] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:04 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:05.388557 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-016aafe1-68c0-43a2-83db-cf267b5c5d57 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:05.389187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-016aafe1-68c0-43a2-83db-cf267b5c5d57 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:05.395968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-016aafe1-68c0-43a2-83db-cf267b5c5d57 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:05.396406 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-016aafe1-68c0-43a2-83db-cf267b5c5d57 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:05.397186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-016aafe1-68c0-43a2-83db-cf267b5c5d57 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:05.397613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-016aafe1-68c0-43a2-83db-cf267b5c5d57 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:05.462715 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-016aafe1-68c0-43a2-83db-cf267b5c5d57 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:05.473806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-016aafe1-68c0-43a2-83db-cf267b5c5d57 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.089387 Jun 10 13:22:05.474437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 885/1687] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:05 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:05.798594 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ebb30c2f-0f53-40fc-9326-de7c3903c489 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:05.799201 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ebb30c2f-0f53-40fc-9326-de7c3903c489 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:05.805765 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ebb30c2f-0f53-40fc-9326-de7c3903c489 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:05.806208 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ebb30c2f-0f53-40fc-9326-de7c3903c489 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:05.806957 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ebb30c2f-0f53-40fc-9326-de7c3903c489 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:05.807389 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ebb30c2f-0f53-40fc-9326-de7c3903c489 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:05.935700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ebb30c2f-0f53-40fc-9326-de7c3903c489 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:05.944018 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ebb30c2f-0f53-40fc-9326-de7c3903c489 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.149236 Jun 10 13:22:05.944576 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 803/1688] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:05 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:06.491577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f49f4326-c681-4025-9bbc-ac8fc6bafd5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:06.492178 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f49f4326-c681-4025-9bbc-ac8fc6bafd5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:06.498607 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f49f4326-c681-4025-9bbc-ac8fc6bafd5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:06.499051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f49f4326-c681-4025-9bbc-ac8fc6bafd5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:06.499798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f49f4326-c681-4025-9bbc-ac8fc6bafd5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:06.500225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f49f4326-c681-4025-9bbc-ac8fc6bafd5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:06.558614 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f49f4326-c681-4025-9bbc-ac8fc6bafd5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:06.566444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f49f4326-c681-4025-9bbc-ac8fc6bafd5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.078942 Jun 10 13:22:06.566998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 886/1689] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:06 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:06.961252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a1578a35-0f1c-45d8-b7e5-11a1b222ac01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:06.961859 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a1578a35-0f1c-45d8-b7e5-11a1b222ac01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:06.968337 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a1578a35-0f1c-45d8-b7e5-11a1b222ac01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:06.968804 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a1578a35-0f1c-45d8-b7e5-11a1b222ac01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:06.969568 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a1578a35-0f1c-45d8-b7e5-11a1b222ac01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:06.970000 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a1578a35-0f1c-45d8-b7e5-11a1b222ac01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:07.093286 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a1578a35-0f1c-45d8-b7e5-11a1b222ac01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:07.102094 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a1578a35-0f1c-45d8-b7e5-11a1b222ac01 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.144757 Jun 10 13:22:07.102657 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 804/1690] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:06 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:07.583219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-33e0af4b-4fa1-4456-a3f2-44d1991d82a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:07.583745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-33e0af4b-4fa1-4456-a3f2-44d1991d82a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:07.590226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33e0af4b-4fa1-4456-a3f2-44d1991d82a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:07.590664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33e0af4b-4fa1-4456-a3f2-44d1991d82a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:07.591414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33e0af4b-4fa1-4456-a3f2-44d1991d82a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:07.591839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33e0af4b-4fa1-4456-a3f2-44d1991d82a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:07.652931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-33e0af4b-4fa1-4456-a3f2-44d1991d82a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:07.660749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-33e0af4b-4fa1-4456-a3f2-44d1991d82a7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.081475 Jun 10 13:22:07.661319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 887/1691] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:07 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:08.118249 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2dc2c1ad-8b35-4af8-a9c5-fd082fb781d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:08.118855 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2dc2c1ad-8b35-4af8-a9c5-fd082fb781d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:08.125471 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dc2c1ad-8b35-4af8-a9c5-fd082fb781d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:08.125915 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dc2c1ad-8b35-4af8-a9c5-fd082fb781d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:08.126668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dc2c1ad-8b35-4af8-a9c5-fd082fb781d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:08.127097 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dc2c1ad-8b35-4af8-a9c5-fd082fb781d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:08.254722 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2dc2c1ad-8b35-4af8-a9c5-fd082fb781d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:08.263008 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2dc2c1ad-8b35-4af8-a9c5-fd082fb781d3 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.148166 Jun 10 13:22:08.263118 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 805/1692] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:08 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:08.677988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fb680fad-697b-4b29-9a99-cd1a96b9955e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:08.678589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fb680fad-697b-4b29-9a99-cd1a96b9955e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:08.685030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb680fad-697b-4b29-9a99-cd1a96b9955e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:08.685476 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb680fad-697b-4b29-9a99-cd1a96b9955e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:08.686219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb680fad-697b-4b29-9a99-cd1a96b9955e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:08.686649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb680fad-697b-4b29-9a99-cd1a96b9955e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:08.704427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:22:08.746532 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fb680fad-697b-4b29-9a99-cd1a96b9955e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:08.754635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fb680fad-697b-4b29-9a99-cd1a96b9955e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080951 Jun 10 13:22:08.755199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 888/1693] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:08 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:09.280176 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2dfedfb0-e8bb-47cb-a008-a3fa5c49b19a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:09.280791 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2dfedfb0-e8bb-47cb-a008-a3fa5c49b19a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:09.287273 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dfedfb0-e8bb-47cb-a008-a3fa5c49b19a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:09.287707 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dfedfb0-e8bb-47cb-a008-a3fa5c49b19a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:09.288453 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dfedfb0-e8bb-47cb-a008-a3fa5c49b19a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:09.288913 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dfedfb0-e8bb-47cb-a008-a3fa5c49b19a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:09.413801 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2dfedfb0-e8bb-47cb-a008-a3fa5c49b19a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:09.421509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2dfedfb0-e8bb-47cb-a008-a3fa5c49b19a tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145351 Jun 10 13:22:09.422066 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 806/1694] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:09 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:09.772155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9cb77661-7564-46e0-b5bb-aa0d0fcc86f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:09.772913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9cb77661-7564-46e0-b5bb-aa0d0fcc86f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:09.779821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cb77661-7564-46e0-b5bb-aa0d0fcc86f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:09.780259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cb77661-7564-46e0-b5bb-aa0d0fcc86f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:09.781056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cb77661-7564-46e0-b5bb-aa0d0fcc86f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:09.781485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cb77661-7564-46e0-b5bb-aa0d0fcc86f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:09.841912 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9cb77661-7564-46e0-b5bb-aa0d0fcc86f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:09.850027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9cb77661-7564-46e0-b5bb-aa0d0fcc86f0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082369 Jun 10 13:22:09.850592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 889/1695] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:09 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:10.438426 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2e44a57f-efd9-401e-aa2e-8561272f0ebf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:10.439023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2e44a57f-efd9-401e-aa2e-8561272f0ebf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:10.445495 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2e44a57f-efd9-401e-aa2e-8561272f0ebf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:10.445940 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2e44a57f-efd9-401e-aa2e-8561272f0ebf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:10.446692 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2e44a57f-efd9-401e-aa2e-8561272f0ebf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:10.447118 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2e44a57f-efd9-401e-aa2e-8561272f0ebf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:10.572668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2e44a57f-efd9-401e-aa2e-8561272f0ebf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:10.580530 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2e44a57f-efd9-401e-aa2e-8561272f0ebf tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145933 Jun 10 13:22:10.581115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 807/1696] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:10 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:10.867319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a56bc739-7c5f-46c1-8c04-4f8164337acc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:10.867854 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a56bc739-7c5f-46c1-8c04-4f8164337acc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:10.874255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a56bc739-7c5f-46c1-8c04-4f8164337acc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:10.874695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a56bc739-7c5f-46c1-8c04-4f8164337acc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:10.875445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a56bc739-7c5f-46c1-8c04-4f8164337acc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:10.875873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a56bc739-7c5f-46c1-8c04-4f8164337acc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:10.935994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a56bc739-7c5f-46c1-8c04-4f8164337acc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:10.943526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a56bc739-7c5f-46c1-8c04-4f8164337acc tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080112 Jun 10 13:22:10.944126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 890/1697] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:10 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:11.598207 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-97bdcb1d-e5a5-4ba3-b151-8afe2c180a14 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:11.598933 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-97bdcb1d-e5a5-4ba3-b151-8afe2c180a14 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:11.605777 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-97bdcb1d-e5a5-4ba3-b151-8afe2c180a14 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:11.606224 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-97bdcb1d-e5a5-4ba3-b151-8afe2c180a14 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:11.606986 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-97bdcb1d-e5a5-4ba3-b151-8afe2c180a14 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:11.607420 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-97bdcb1d-e5a5-4ba3-b151-8afe2c180a14 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:11.731389 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-97bdcb1d-e5a5-4ba3-b151-8afe2c180a14 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:11.739277 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-97bdcb1d-e5a5-4ba3-b151-8afe2c180a14 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.145560 Jun 10 13:22:11.739838 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 808/1698] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:11 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:11.960146 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8f8e65ba-12f8-465e-991e-56a36f60d1a0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:11.960749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8f8e65ba-12f8-465e-991e-56a36f60d1a0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:11.967259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f8e65ba-12f8-465e-991e-56a36f60d1a0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:11.967699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f8e65ba-12f8-465e-991e-56a36f60d1a0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:11.968445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f8e65ba-12f8-465e-991e-56a36f60d1a0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:11.968909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f8e65ba-12f8-465e-991e-56a36f60d1a0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:12.036184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8f8e65ba-12f8-465e-991e-56a36f60d1a0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:12.043897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8f8e65ba-12f8-465e-991e-56a36f60d1a0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.087640 Jun 10 13:22:12.044468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 891/1699] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:11 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:12.755798 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a4c2cf47-2e55-436c-ad73-caa9b36ed891 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:12.756407 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a4c2cf47-2e55-436c-ad73-caa9b36ed891 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:12.762929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a4c2cf47-2e55-436c-ad73-caa9b36ed891 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:12.763373 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a4c2cf47-2e55-436c-ad73-caa9b36ed891 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:12.764127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a4c2cf47-2e55-436c-ad73-caa9b36ed891 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:12.764557 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a4c2cf47-2e55-436c-ad73-caa9b36ed891 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:12.894795 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a4c2cf47-2e55-436c-ad73-caa9b36ed891 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:12.902610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a4c2cf47-2e55-436c-ad73-caa9b36ed891 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.151164 Jun 10 13:22:12.903171 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 809/1700] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:12 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:13.061123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e8ffdf4a-f1b6-4cbc-a0b4-e517dd4e4f56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:13.061804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e8ffdf4a-f1b6-4cbc-a0b4-e517dd4e4f56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:13.068277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8ffdf4a-f1b6-4cbc-a0b4-e517dd4e4f56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:13.068727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8ffdf4a-f1b6-4cbc-a0b4-e517dd4e4f56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:13.069524 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8ffdf4a-f1b6-4cbc-a0b4-e517dd4e4f56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:13.069989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8ffdf4a-f1b6-4cbc-a0b4-e517dd4e4f56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:13.129027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e8ffdf4a-f1b6-4cbc-a0b4-e517dd4e4f56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:13.136433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e8ffdf4a-f1b6-4cbc-a0b4-e517dd4e4f56 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079408 Jun 10 13:22:13.137030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 892/1701] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:13 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:22:13.968952 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-abeae440-484c-4d45-b933-7ebbe5eb26eb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:13.969562 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-abeae440-484c-4d45-b933-7ebbe5eb26eb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:13.976378 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-abeae440-484c-4d45-b933-7ebbe5eb26eb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:13.976858 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-abeae440-484c-4d45-b933-7ebbe5eb26eb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:13.977621 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-abeae440-484c-4d45-b933-7ebbe5eb26eb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:13.978052 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-abeae440-484c-4d45-b933-7ebbe5eb26eb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:14.105044 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-abeae440-484c-4d45-b933-7ebbe5eb26eb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:14.113045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-abeae440-484c-4d45-b933-7ebbe5eb26eb tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.197483 Jun 10 13:22:14.113610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 810/1702] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:13 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:14.153428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c9167817-088e-418e-b164-b3a2553412b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:14.154036 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c9167817-088e-418e-b164-b3a2553412b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:14.156345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=56 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:22:14.161709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9167817-088e-418e-b164-b3a2553412b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:14.162159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9167817-088e-418e-b164-b3a2553412b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:14.162960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9167817-088e-418e-b164-b3a2553412b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:14.163387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9167817-088e-418e-b164-b3a2553412b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:14.222796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c9167817-088e-418e-b164-b3a2553412b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:14.230415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c9167817-088e-418e-b164-b3a2553412b6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080842 Jun 10 13:22:14.231052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 893/1703] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:14 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:15.130224 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7a960532-6389-4b27-8c47-5e4a11c12097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:15.130803 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7a960532-6389-4b27-8c47-5e4a11c12097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:15.137247 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7a960532-6389-4b27-8c47-5e4a11c12097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:15.137686 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7a960532-6389-4b27-8c47-5e4a11c12097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:15.138461 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7a960532-6389-4b27-8c47-5e4a11c12097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:15.138889 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7a960532-6389-4b27-8c47-5e4a11c12097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:15.247138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4000c5c7-e58f-4602-ac52-f34175a78213 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:15.247827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4000c5c7-e58f-4602-ac52-f34175a78213 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:15.254333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4000c5c7-e58f-4602-ac52-f34175a78213 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:15.254785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4000c5c7-e58f-4602-ac52-f34175a78213 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:15.255540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4000c5c7-e58f-4602-ac52-f34175a78213 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:15.255965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4000c5c7-e58f-4602-ac52-f34175a78213 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:15.264941 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7a960532-6389-4b27-8c47-5e4a11c12097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:15.268293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:22:15.273116 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7a960532-6389-4b27-8c47-5e4a11c12097 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.146733 Jun 10 13:22:15.273684 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 811/1704] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:15 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:15.315832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4000c5c7-e58f-4602-ac52-f34175a78213 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:15.323869 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4000c5c7-e58f-4602-ac52-f34175a78213 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080540 Jun 10 13:22:15.324436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 894/1705] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:15 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:22:16.290794 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2dcc5089-0369-4d27-9ff8-d434c84db7a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:16.291409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2dcc5089-0369-4d27-9ff8-d434c84db7a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: c699e532-a2dd-4fb8-9ba9-0e7089eff32f] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:16.299086 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dcc5089-0369-4d27-9ff8-d434c84db7a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:16.300048 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dcc5089-0369-4d27-9ff8-d434c84db7a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:16.300838 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dcc5089-0369-4d27-9ff8-d434c84db7a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:16.301280 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2dcc5089-0369-4d27-9ff8-d434c84db7a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:16.323137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-334c830a-80f2-44b3-bdb0-11821c7c4be7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:16.324388 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-334c830a-80f2-44b3-bdb0-11821c7c4be7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:16.331163 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-334c830a-80f2-44b3-bdb0-11821c7c4be7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:16.331920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-334c830a-80f2-44b3-bdb0-11821c7c4be7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:16.332972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-334c830a-80f2-44b3-bdb0-11821c7c4be7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:16.333623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-334c830a-80f2-44b3-bdb0-11821c7c4be7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:16.380319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-334c830a-80f2-44b3-bdb0-11821c7c4be7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:22:16.458884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2dcc5089-0369-4d27-9ff8-d434c84db7a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:16.467323 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2dcc5089-0369-4d27-9ff8-d434c84db7a8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f" status: 200 len: 1734 microversion: 2.1 time: 0.180334 Jun 10 13:22:16.467895 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 812/1706] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:16 2022] GET /compute/v2.1/servers/c699e532-a2dd-4fb8-9ba9-0e7089eff32f => generated 1734 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:16.473953 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-8f33631f-383e-4417-80fd-3f22a91cc444 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:16.474367 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-8f33631f-383e-4417-80fd-3f22a91cc444 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:16.480856 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8f33631f-383e-4417-80fd-3f22a91cc444 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:16.481308 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8f33631f-383e-4417-80fd-3f22a91cc444 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:16.482080 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8f33631f-383e-4417-80fd-3f22a91cc444 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:16.482513 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-8f33631f-383e-4417-80fd-3f22a91cc444 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:16.543728 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-8f33631f-383e-4417-80fd-3f22a91cc444 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:16.551775 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-8f33631f-383e-4417-80fd-3f22a91cc444 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082742 Jun 10 13:22:16.556724 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 813/1707] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:16 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:16.556724 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-542c9eb3-e073-447e-b2e9-0764ee5c6c74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '.version_select at 0x7f9309ab2f70>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:16.599405 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-334c830a-80f2-44b3-bdb0-11821c7c4be7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "DELETE /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 204 len: 0 microversion: 2.20 time: 0.279775 Jun 10 13:22:16.599993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 895/1708] 10.222.0.25 () {60 vars in 1279 bytes} [Fri Jun 10 13:22:16 2022] DELETE /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Jun 10 13:22:16.608578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-78a327cd-1ee2-41bf-8011-4d8c98022528 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:16.609295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-78a327cd-1ee2-41bf-8011-4d8c98022528 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:16.615839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78a327cd-1ee2-41bf-8011-4d8c98022528 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:16.616277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78a327cd-1ee2-41bf-8011-4d8c98022528 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:16.617092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78a327cd-1ee2-41bf-8011-4d8c98022528 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:16.617517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78a327cd-1ee2-41bf-8011-4d8c98022528 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:16.629092 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-542c9eb3-e073-447e-b2e9-0764ee5c6c74 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-80950017" status: 202 len: 0 microversion: 2.1 time: 0.075617 Jun 10 13:22:16.629900 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 814/1709] 10.222.0.25 () {58 vars in 1214 bytes} [Fri Jun 10 13:22:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-80950017 => generated 0 bytes in 77 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jun 10 13:22:16.774683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-78a327cd-1ee2-41bf-8011-4d8c98022528 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:16.783513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-78a327cd-1ee2-41bf-8011-4d8c98022528 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:16.785724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-78a327cd-1ee2-41bf-8011-4d8c98022528 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:16.786987 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-78a327cd-1ee2-41bf-8011-4d8c98022528 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1800 microversion: 2.20 time: 0.182485 Jun 10 13:22:16.787879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 896/1710] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:16 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1800 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:22:17.570189 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-aed96a36-0ff8-4936-a165-8cf9208c0d79 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:17.570813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-aed96a36-0ff8-4936-a165-8cf9208c0d79 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:17.572734 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=57,nova_api:DELETE=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:22:17.578795 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-aed96a36-0ff8-4936-a165-8cf9208c0d79 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:17.579233 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-aed96a36-0ff8-4936-a165-8cf9208c0d79 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:17.580068 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-aed96a36-0ff8-4936-a165-8cf9208c0d79 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:17.580736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-aed96a36-0ff8-4936-a165-8cf9208c0d79 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:17.806640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2113c0b3-66ae-4965-85b5-a518f6be7dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:17.807368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2113c0b3-66ae-4965-85b5-a518f6be7dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:17.808650 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-aed96a36-0ff8-4936-a165-8cf9208c0d79 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:17.814027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2113c0b3-66ae-4965-85b5-a518f6be7dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:17.814759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2113c0b3-66ae-4965-85b5-a518f6be7dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:17.816045 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2113c0b3-66ae-4965-85b5-a518f6be7dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:17.817116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2113c0b3-66ae-4965-85b5-a518f6be7dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:17.818694 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-aed96a36-0ff8-4936-a165-8cf9208c0d79 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.253661 Jun 10 13:22:17.819293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 815/1711] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:17 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:17.932724 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-73a19ca2-e497-4ea4-8a20-b419c53e9d2b req-e1eaba57-bec1-4fa2-973c-d2013ee426eb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c699e532-a2dd-4fb8-9ba9-0e7089eff32f", "tag": "9d09560d-d177-4fe5-bce5-a1dc351d5f14"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:22:17.941528 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-73a19ca2-e497-4ea4-8a20-b419c53e9d2b req-e1eaba57-bec1-4fa2-973c-d2013ee426eb service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:17.941966 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-73a19ca2-e497-4ea4-8a20-b419c53e9d2b req-e1eaba57-bec1-4fa2-973c-d2013ee426eb service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:17.952883 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-73a19ca2-e497-4ea4-8a20-b419c53e9d2b req-e1eaba57-bec1-4fa2-973c-d2013ee426eb service nova] Creating event network-changed:9d09560d-d177-4fe5-bce5-a1dc351d5f14 for instance c699e532-a2dd-4fb8-9ba9-0e7089eff32f on hv-cinder-maste Jun 10 13:22:17.958509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-73a19ca2-e497-4ea4-8a20-b419c53e9d2b req-e1eaba57-bec1-4fa2-973c-d2013ee426eb service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083158 Jun 10 13:22:17.959448 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 816/1712] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:22:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:22:17.968907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2113c0b3-66ae-4965-85b5-a518f6be7dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:17.978380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2113c0b3-66ae-4965-85b5-a518f6be7dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:17.979967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2113c0b3-66ae-4965-85b5-a518f6be7dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:17.981164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2113c0b3-66ae-4965-85b5-a518f6be7dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1800 microversion: 2.20 time: 0.178628 Jun 10 13:22:17.981952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 897/1713] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:17 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1800 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:22:18.836485 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b06660ed-a486-491e-82fb-bef7f62b9e5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:18.838543 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b06660ed-a486-491e-82fb-bef7f62b9e5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:18.845274 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b06660ed-a486-491e-82fb-bef7f62b9e5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:18.845274 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b06660ed-a486-491e-82fb-bef7f62b9e5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:18.847309 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b06660ed-a486-491e-82fb-bef7f62b9e5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:18.847309 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b06660ed-a486-491e-82fb-bef7f62b9e5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:18.954040 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b06660ed-a486-491e-82fb-bef7f62b9e5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:18.962975 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b06660ed-a486-491e-82fb-bef7f62b9e5b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.131255 Jun 10 13:22:18.963633 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 817/1714] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:18 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:22:18.999891 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-89cc449c-2fb7-4da6-bc2b-647053928df1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:19.000682 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-89cc449c-2fb7-4da6-bc2b-647053928df1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:19.007315 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-89cc449c-2fb7-4da6-bc2b-647053928df1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:19.007841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-89cc449c-2fb7-4da6-bc2b-647053928df1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:19.008683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-89cc449c-2fb7-4da6-bc2b-647053928df1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:19.009288 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-89cc449c-2fb7-4da6-bc2b-647053928df1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:19.152543 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-89cc449c-2fb7-4da6-bc2b-647053928df1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:19.162187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-89cc449c-2fb7-4da6-bc2b-647053928df1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:19.163749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-89cc449c-2fb7-4da6-bc2b-647053928df1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:19.165539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-89cc449c-2fb7-4da6-bc2b-647053928df1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1800 microversion: 2.20 time: 0.168938 Jun 10 13:22:19.166104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 898/1715] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:18 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1800 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:19.974629 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [req-f7b75a11-c015-492c-afab-a61e70430c35 req-acbfb97c-8f98-4a1d-a665-eb3110f79093 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c699e532-a2dd-4fb8-9ba9-0e7089eff32f", "name": "network-vif-deleted", "tag": "9d09560d-d177-4fe5-bce5-a1dc351d5f14"}, {"server_uuid": "db56898a-dff9-4cad-bf8d-f84b53247fc3", "name": "network-vif-deleted", "tag": "31e1a9d0-3f1e-4221-a431-639f0e4a6ff4"}]} {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:22:19.983177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a201563b-0780-488f-b6df-e40d1eb40ad4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:19.983935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a201563b-0780-488f-b6df-e40d1eb40ad4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:19.984044 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-f7b75a11-c015-492c-afab-a61e70430c35 req-acbfb97c-8f98-4a1d-a665-eb3110f79093 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:19.984397 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [req-f7b75a11-c015-492c-afab-a61e70430c35 req-acbfb97c-8f98-4a1d-a665-eb3110f79093 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:19.990270 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a201563b-0780-488f-b6df-e40d1eb40ad4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:19.990812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a201563b-0780-488f-b6df-e40d1eb40ad4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:19.991643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a201563b-0780-488f-b6df-e40d1eb40ad4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:19.992142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a201563b-0780-488f-b6df-e40d1eb40ad4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:20.062183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a201563b-0780-488f-b6df-e40d1eb40ad4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:20.072077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a201563b-0780-488f-b6df-e40d1eb40ad4 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.092355 Jun 10 13:22:20.083070 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 899/1716] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:19 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:22:20.150076 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-f7b75a11-c015-492c-afab-a61e70430c35 req-acbfb97c-8f98-4a1d-a665-eb3110f79093 service nova] Creating event network-vif-deleted:9d09560d-d177-4fe5-bce5-a1dc351d5f14 for instance c699e532-a2dd-4fb8-9ba9-0e7089eff32f on hv-cinder-maste Jun 10 13:22:20.150578 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.compute.server_external_events [req-f7b75a11-c015-492c-afab-a61e70430c35 req-acbfb97c-8f98-4a1d-a665-eb3110f79093 service nova] Creating event network-vif-deleted:31e1a9d0-3f1e-4221-a431-639f0e4a6ff4 for instance db56898a-dff9-4cad-bf8d-f84b53247fc3 on hv-cinder-maste Jun 10 13:22:20.157041 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [req-f7b75a11-c015-492c-afab-a61e70430c35 req-acbfb97c-8f98-4a1d-a665-eb3110f79093 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.186628 Jun 10 13:22:20.157654 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 818/1717] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:22:19 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:22:20.183597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8d544994-963d-423d-a43d-bea2dffb58a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:20.184240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8d544994-963d-423d-a43d-bea2dffb58a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:20.190878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8d544994-963d-423d-a43d-bea2dffb58a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:20.191324 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8d544994-963d-423d-a43d-bea2dffb58a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:20.192831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8d544994-963d-423d-a43d-bea2dffb58a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:20.193301 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8d544994-963d-423d-a43d-bea2dffb58a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:20.257510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8d544994-963d-423d-a43d-bea2dffb58a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:20.269105 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8d544994-963d-423d-a43d-bea2dffb58a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:20.270318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8d544994-963d-423d-a43d-bea2dffb58a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:20.271568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8d544994-963d-423d-a43d-bea2dffb58a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.091904 Jun 10 13:22:20.272264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 900/1718] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:20 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:21.091509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c1c8fed2-261a-4cdf-8de4-2a141a5984b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:21.092210 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c1c8fed2-261a-4cdf-8de4-2a141a5984b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:21.098955 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c1c8fed2-261a-4cdf-8de4-2a141a5984b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:21.099411 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c1c8fed2-261a-4cdf-8de4-2a141a5984b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:21.100167 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c1c8fed2-261a-4cdf-8de4-2a141a5984b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:21.100871 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c1c8fed2-261a-4cdf-8de4-2a141a5984b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:21.163266 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c1c8fed2-261a-4cdf-8de4-2a141a5984b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:21.170768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c1c8fed2-261a-4cdf-8de4-2a141a5984b8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.083961 Jun 10 13:22:21.171328 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 819/1719] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:21 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:21.290194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ffd1cf8b-c812-4e32-a3c1-7d0b2c19e0d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:21.290799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ffd1cf8b-c812-4e32-a3c1-7d0b2c19e0d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:21.298853 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ffd1cf8b-c812-4e32-a3c1-7d0b2c19e0d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:21.299911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ffd1cf8b-c812-4e32-a3c1-7d0b2c19e0d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:21.300682 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ffd1cf8b-c812-4e32-a3c1-7d0b2c19e0d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:21.301136 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ffd1cf8b-c812-4e32-a3c1-7d0b2c19e0d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:21.362706 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ffd1cf8b-c812-4e32-a3c1-7d0b2c19e0d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:21.371235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ffd1cf8b-c812-4e32-a3c1-7d0b2c19e0d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:21.372724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ffd1cf8b-c812-4e32-a3c1-7d0b2c19e0d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:21.373888 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ffd1cf8b-c812-4e32-a3c1-7d0b2c19e0d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087664 Jun 10 13:22:21.374455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 901/1720] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:21 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:22.188180 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1697e733-e8ea-4a63-90bb-e2d50447ff9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:22.188713 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1697e733-e8ea-4a63-90bb-e2d50447ff9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:22.195224 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1697e733-e8ea-4a63-90bb-e2d50447ff9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:22.195667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1697e733-e8ea-4a63-90bb-e2d50447ff9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:22.196427 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1697e733-e8ea-4a63-90bb-e2d50447ff9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:22.196888 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1697e733-e8ea-4a63-90bb-e2d50447ff9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:22.255984 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1697e733-e8ea-4a63-90bb-e2d50447ff9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:22.263357 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1697e733-e8ea-4a63-90bb-e2d50447ff9f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079179 Jun 10 13:22:22.263909 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 820/1721] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:22 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:22.391403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-85a1cb5f-4602-46ea-8312-c11a767e4980 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:22.392003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-85a1cb5f-4602-46ea-8312-c11a767e4980 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:22.398587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-85a1cb5f-4602-46ea-8312-c11a767e4980 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:22.399030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-85a1cb5f-4602-46ea-8312-c11a767e4980 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:22.399791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-85a1cb5f-4602-46ea-8312-c11a767e4980 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:22.400217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-85a1cb5f-4602-46ea-8312-c11a767e4980 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:22.465052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-85a1cb5f-4602-46ea-8312-c11a767e4980 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:22.474154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-85a1cb5f-4602-46ea-8312-c11a767e4980 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:22.475898 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-85a1cb5f-4602-46ea-8312-c11a767e4980 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:22.477065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-85a1cb5f-4602-46ea-8312-c11a767e4980 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.089548 Jun 10 13:22:22.477616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 902/1722] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:22 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:23.280598 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ef07b01b-c80c-44d8-a658-1a2bce728b08 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:23.281225 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ef07b01b-c80c-44d8-a658-1a2bce728b08 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:23.287640 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef07b01b-c80c-44d8-a658-1a2bce728b08 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:23.288079 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef07b01b-c80c-44d8-a658-1a2bce728b08 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:23.288855 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef07b01b-c80c-44d8-a658-1a2bce728b08 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:23.289287 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ef07b01b-c80c-44d8-a658-1a2bce728b08 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:23.351812 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ef07b01b-c80c-44d8-a658-1a2bce728b08 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:23.359181 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ef07b01b-c80c-44d8-a658-1a2bce728b08 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082451 Jun 10 13:22:23.359749 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 821/1723] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:23 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:23.495400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9a905daa-5d91-4fc4-a226-4dbd632ed757 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:23.496010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9a905daa-5d91-4fc4-a226-4dbd632ed757 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:23.503230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a905daa-5d91-4fc4-a226-4dbd632ed757 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:23.503677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a905daa-5d91-4fc4-a226-4dbd632ed757 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:23.504433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a905daa-5d91-4fc4-a226-4dbd632ed757 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:23.504922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a905daa-5d91-4fc4-a226-4dbd632ed757 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:23.522907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:22:23.564311 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9a905daa-5d91-4fc4-a226-4dbd632ed757 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:23.573207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9a905daa-5d91-4fc4-a226-4dbd632ed757 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:23.574709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9a905daa-5d91-4fc4-a226-4dbd632ed757 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:23.575858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9a905daa-5d91-4fc4-a226-4dbd632ed757 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084398 Jun 10 13:22:23.576421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 903/1724] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:23 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:24.375518 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-03d309b9-ad87-4158-946b-ca131c4c5260 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:24.376135 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-03d309b9-ad87-4158-946b-ca131c4c5260 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:24.382716 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-03d309b9-ad87-4158-946b-ca131c4c5260 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:24.383161 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-03d309b9-ad87-4158-946b-ca131c4c5260 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:24.383924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-03d309b9-ad87-4158-946b-ca131c4c5260 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:24.384356 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-03d309b9-ad87-4158-946b-ca131c4c5260 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:24.446216 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-03d309b9-ad87-4158-946b-ca131c4c5260 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:24.453789 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-03d309b9-ad87-4158-946b-ca131c4c5260 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082102 Jun 10 13:22:24.454339 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 822/1725] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:24 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:24.593595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6b8121b0-06c6-41bd-b2c2-0afe8c6db88f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:24.594117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6b8121b0-06c6-41bd-b2c2-0afe8c6db88f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:24.602022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6b8121b0-06c6-41bd-b2c2-0afe8c6db88f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:24.602464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6b8121b0-06c6-41bd-b2c2-0afe8c6db88f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:24.603211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6b8121b0-06c6-41bd-b2c2-0afe8c6db88f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:24.603636 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6b8121b0-06c6-41bd-b2c2-0afe8c6db88f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:24.662644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6b8121b0-06c6-41bd-b2c2-0afe8c6db88f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:24.670859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6b8121b0-06c6-41bd-b2c2-0afe8c6db88f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:24.672336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6b8121b0-06c6-41bd-b2c2-0afe8c6db88f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:24.673501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6b8121b0-06c6-41bd-b2c2-0afe8c6db88f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084458 Jun 10 13:22:24.674085 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 904/1726] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:24 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:25.471190 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-41c95c73-95e6-4a87-a927-f256dc3ce219 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:25.471832 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-41c95c73-95e6-4a87-a927-f256dc3ce219 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:25.478697 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-41c95c73-95e6-4a87-a927-f256dc3ce219 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:25.479141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-41c95c73-95e6-4a87-a927-f256dc3ce219 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:25.479895 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-41c95c73-95e6-4a87-a927-f256dc3ce219 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:25.480327 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-41c95c73-95e6-4a87-a927-f256dc3ce219 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:25.539151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-41c95c73-95e6-4a87-a927-f256dc3ce219 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:25.546434 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-41c95c73-95e6-4a87-a927-f256dc3ce219 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079173 Jun 10 13:22:25.546987 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 823/1727] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:25 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:25.691247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0f2d1656-defd-49ee-b29e-c9c1b94fcefb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:25.691850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0f2d1656-defd-49ee-b29e-c9c1b94fcefb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:25.698281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f2d1656-defd-49ee-b29e-c9c1b94fcefb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:25.698726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f2d1656-defd-49ee-b29e-c9c1b94fcefb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:25.699475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f2d1656-defd-49ee-b29e-c9c1b94fcefb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:25.699905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f2d1656-defd-49ee-b29e-c9c1b94fcefb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:25.758390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0f2d1656-defd-49ee-b29e-c9c1b94fcefb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:25.766548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0f2d1656-defd-49ee-b29e-c9c1b94fcefb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:25.768072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0f2d1656-defd-49ee-b29e-c9c1b94fcefb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:25.769241 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0f2d1656-defd-49ee-b29e-c9c1b94fcefb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.081888 Jun 10 13:22:25.769793 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 905/1728] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:25 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:26.563796 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-09469bcd-1ac4-4918-afcb-088faf797814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:26.564358 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-09469bcd-1ac4-4918-afcb-088faf797814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:26.571491 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-09469bcd-1ac4-4918-afcb-088faf797814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:26.571935 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-09469bcd-1ac4-4918-afcb-088faf797814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:26.572694 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-09469bcd-1ac4-4918-afcb-088faf797814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:26.573155 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-09469bcd-1ac4-4918-afcb-088faf797814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:26.633744 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-09469bcd-1ac4-4918-afcb-088faf797814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:26.641068 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-09469bcd-1ac4-4918-afcb-088faf797814 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.081169 Jun 10 13:22:26.641620 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 824/1729] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:26 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:26.786834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c7955a44-604c-42e2-beae-4f55e19ee5af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:26.787439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c7955a44-604c-42e2-beae-4f55e19ee5af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:26.793892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7955a44-604c-42e2-beae-4f55e19ee5af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:26.794338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7955a44-604c-42e2-beae-4f55e19ee5af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:26.795092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7955a44-604c-42e2-beae-4f55e19ee5af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:26.795521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7955a44-604c-42e2-beae-4f55e19ee5af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:26.857959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c7955a44-604c-42e2-beae-4f55e19ee5af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:26.867024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c7955a44-604c-42e2-beae-4f55e19ee5af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:26.868534 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c7955a44-604c-42e2-beae-4f55e19ee5af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:26.869690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c7955a44-604c-42e2-beae-4f55e19ee5af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086745 Jun 10 13:22:26.870233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 906/1730] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:26 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:27.658219 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f072821d-dc31-40d6-864e-9167c1cb59e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:27.658828 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f072821d-dc31-40d6-864e-9167c1cb59e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:27.666121 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f072821d-dc31-40d6-864e-9167c1cb59e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:27.666567 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f072821d-dc31-40d6-864e-9167c1cb59e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:27.667329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f072821d-dc31-40d6-864e-9167c1cb59e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:27.667761 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f072821d-dc31-40d6-864e-9167c1cb59e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:27.728635 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f072821d-dc31-40d6-864e-9167c1cb59e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:27.735946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f072821d-dc31-40d6-864e-9167c1cb59e6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.081566 Jun 10 13:22:27.736492 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 825/1731] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:27 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:27.887988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e02cfd45-2189-4fdf-9258-20ff123ae460 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:27.888592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e02cfd45-2189-4fdf-9258-20ff123ae460 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:27.895658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e02cfd45-2189-4fdf-9258-20ff123ae460 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:27.896104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e02cfd45-2189-4fdf-9258-20ff123ae460 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:27.896890 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e02cfd45-2189-4fdf-9258-20ff123ae460 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:27.897347 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e02cfd45-2189-4fdf-9258-20ff123ae460 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:27.963802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e02cfd45-2189-4fdf-9258-20ff123ae460 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:27.972192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e02cfd45-2189-4fdf-9258-20ff123ae460 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:27.973717 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e02cfd45-2189-4fdf-9258-20ff123ae460 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:27.974854 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e02cfd45-2189-4fdf-9258-20ff123ae460 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.090827 Jun 10 13:22:27.975672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 907/1732] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:27 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:28.753334 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-69a14884-386b-4d3a-ba33-4a8dcecca8da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:28.753936 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-69a14884-386b-4d3a-ba33-4a8dcecca8da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:28.760463 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69a14884-386b-4d3a-ba33-4a8dcecca8da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:28.760941 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69a14884-386b-4d3a-ba33-4a8dcecca8da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:28.761949 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69a14884-386b-4d3a-ba33-4a8dcecca8da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:28.762260 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69a14884-386b-4d3a-ba33-4a8dcecca8da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:28.822752 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-69a14884-386b-4d3a-ba33-4a8dcecca8da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:28.830616 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-69a14884-386b-4d3a-ba33-4a8dcecca8da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.081716 Jun 10 13:22:28.831235 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 826/1733] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:28 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:28.993024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cf2b159f-c715-4b80-8483-5d305bb933b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:28.993631 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cf2b159f-c715-4b80-8483-5d305bb933b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:29.000205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf2b159f-c715-4b80-8483-5d305bb933b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:29.000652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf2b159f-c715-4b80-8483-5d305bb933b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:29.001438 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf2b159f-c715-4b80-8483-5d305bb933b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:29.001876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf2b159f-c715-4b80-8483-5d305bb933b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:29.061149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cf2b159f-c715-4b80-8483-5d305bb933b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:29.069421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cf2b159f-c715-4b80-8483-5d305bb933b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:29.070936 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cf2b159f-c715-4b80-8483-5d305bb933b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:29.072051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cf2b159f-c715-4b80-8483-5d305bb933b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082945 Jun 10 13:22:29.072598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 908/1734] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:28 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:29.848245 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-457a4deb-49ea-4704-b177-e3abc28951ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:29.848877 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-457a4deb-49ea-4704-b177-e3abc28951ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:29.855599 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-457a4deb-49ea-4704-b177-e3abc28951ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:29.856046 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-457a4deb-49ea-4704-b177-e3abc28951ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:29.856838 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-457a4deb-49ea-4704-b177-e3abc28951ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:29.857276 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-457a4deb-49ea-4704-b177-e3abc28951ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:29.915637 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-457a4deb-49ea-4704-b177-e3abc28951ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:29.923006 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-457a4deb-49ea-4704-b177-e3abc28951ba tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079222 Jun 10 13:22:29.923561 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 827/1735] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:29 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:30.089504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a508240e-ec30-4a01-9b21-11c8c1d60d95 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:30.090114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a508240e-ec30-4a01-9b21-11c8c1d60d95 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:30.096533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a508240e-ec30-4a01-9b21-11c8c1d60d95 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:30.097014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a508240e-ec30-4a01-9b21-11c8c1d60d95 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:30.097768 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a508240e-ec30-4a01-9b21-11c8c1d60d95 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:30.098199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a508240e-ec30-4a01-9b21-11c8c1d60d95 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:30.158332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a508240e-ec30-4a01-9b21-11c8c1d60d95 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:30.166627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a508240e-ec30-4a01-9b21-11c8c1d60d95 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:30.168108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a508240e-ec30-4a01-9b21-11c8c1d60d95 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:30.169283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a508240e-ec30-4a01-9b21-11c8c1d60d95 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083760 Jun 10 13:22:30.169839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 909/1736] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:30 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:30.939493 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ac8f2b51-c24b-4714-8105-b3dd056680c8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:30.940073 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ac8f2b51-c24b-4714-8105-b3dd056680c8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:30.946612 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ac8f2b51-c24b-4714-8105-b3dd056680c8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:30.947054 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ac8f2b51-c24b-4714-8105-b3dd056680c8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:30.947800 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ac8f2b51-c24b-4714-8105-b3dd056680c8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:30.948225 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ac8f2b51-c24b-4714-8105-b3dd056680c8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:31.006834 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ac8f2b51-c24b-4714-8105-b3dd056680c8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:31.014583 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ac8f2b51-c24b-4714-8105-b3dd056680c8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079001 Jun 10 13:22:31.015141 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 828/1737] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:30 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:31.187061 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6bfc0de1-76d1-4903-b132-111567e52dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:31.187624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6bfc0de1-76d1-4903-b132-111567e52dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:31.226602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bfc0de1-76d1-4903-b132-111567e52dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:31.226602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bfc0de1-76d1-4903-b132-111567e52dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:31.226602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bfc0de1-76d1-4903-b132-111567e52dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:31.226602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6bfc0de1-76d1-4903-b132-111567e52dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:31.258159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6bfc0de1-76d1-4903-b132-111567e52dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:31.266590 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6bfc0de1-76d1-4903-b132-111567e52dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:31.268614 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6bfc0de1-76d1-4903-b132-111567e52dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:31.269783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6bfc0de1-76d1-4903-b132-111567e52dd4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086681 Jun 10 13:22:31.270341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 910/1738] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:31 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:32.031200 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2555c3e2-6c0d-4cf7-b01c-973c355b1ae6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:32.031807 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2555c3e2-6c0d-4cf7-b01c-973c355b1ae6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:32.038360 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2555c3e2-6c0d-4cf7-b01c-973c355b1ae6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:32.038801 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2555c3e2-6c0d-4cf7-b01c-973c355b1ae6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:32.039550 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2555c3e2-6c0d-4cf7-b01c-973c355b1ae6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:32.039980 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2555c3e2-6c0d-4cf7-b01c-973c355b1ae6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:32.102997 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2555c3e2-6c0d-4cf7-b01c-973c355b1ae6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:32.110522 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2555c3e2-6c0d-4cf7-b01c-973c355b1ae6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.083164 Jun 10 13:22:32.111077 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 829/1739] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:32 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:32.287519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f8ca017e-6f40-40ff-a74d-ae4e9370648e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:32.288188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f8ca017e-6f40-40ff-a74d-ae4e9370648e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:32.294645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8ca017e-6f40-40ff-a74d-ae4e9370648e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:32.295089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8ca017e-6f40-40ff-a74d-ae4e9370648e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:32.295847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8ca017e-6f40-40ff-a74d-ae4e9370648e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:32.296282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8ca017e-6f40-40ff-a74d-ae4e9370648e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:32.356374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f8ca017e-6f40-40ff-a74d-ae4e9370648e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:32.365297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f8ca017e-6f40-40ff-a74d-ae4e9370648e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:32.366790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f8ca017e-6f40-40ff-a74d-ae4e9370648e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:32.367929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f8ca017e-6f40-40ff-a74d-ae4e9370648e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084812 Jun 10 13:22:32.368478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 911/1740] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:32 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:33.127695 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-81f29741-f02d-4870-9591-134f70ae8c87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:33.128300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-81f29741-f02d-4870-9591-134f70ae8c87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:33.134839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-81f29741-f02d-4870-9591-134f70ae8c87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:33.135289 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-81f29741-f02d-4870-9591-134f70ae8c87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:33.136045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-81f29741-f02d-4870-9591-134f70ae8c87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:33.136478 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-81f29741-f02d-4870-9591-134f70ae8c87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:33.194893 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-81f29741-f02d-4870-9591-134f70ae8c87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:33.198317 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:22:33.202570 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-81f29741-f02d-4870-9591-134f70ae8c87 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.078750 Jun 10 13:22:33.203128 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 830/1741] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:33 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:33.386110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-39c53a56-6328-4188-acd3-02c828945ec9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:33.386716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-39c53a56-6328-4188-acd3-02c828945ec9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:33.393324 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39c53a56-6328-4188-acd3-02c828945ec9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:33.393775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39c53a56-6328-4188-acd3-02c828945ec9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:33.394536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39c53a56-6328-4188-acd3-02c828945ec9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:33.394975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39c53a56-6328-4188-acd3-02c828945ec9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:33.453336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-39c53a56-6328-4188-acd3-02c828945ec9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:33.461538 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-39c53a56-6328-4188-acd3-02c828945ec9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:33.463028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-39c53a56-6328-4188-acd3-02c828945ec9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:33.464163 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-39c53a56-6328-4188-acd3-02c828945ec9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.081992 Jun 10 13:22:33.464712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 912/1742] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:33 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:34.219689 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c36b60d7-34b7-4bdb-ad92-b59b40315833 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:34.220295 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c36b60d7-34b7-4bdb-ad92-b59b40315833 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:34.227853 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c36b60d7-34b7-4bdb-ad92-b59b40315833 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:34.228308 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c36b60d7-34b7-4bdb-ad92-b59b40315833 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:34.229096 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c36b60d7-34b7-4bdb-ad92-b59b40315833 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:34.229534 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c36b60d7-34b7-4bdb-ad92-b59b40315833 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:34.290982 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c36b60d7-34b7-4bdb-ad92-b59b40315833 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:34.298502 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c36b60d7-34b7-4bdb-ad92-b59b40315833 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082979 Jun 10 13:22:34.299061 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 831/1743] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:34 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:34.482184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1b7e8803-fd28-405c-9c8f-b83bb1f978f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:34.482710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1b7e8803-fd28-405c-9c8f-b83bb1f978f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:34.489881 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b7e8803-fd28-405c-9c8f-b83bb1f978f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:34.490323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b7e8803-fd28-405c-9c8f-b83bb1f978f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:34.491067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b7e8803-fd28-405c-9c8f-b83bb1f978f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:34.491491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b7e8803-fd28-405c-9c8f-b83bb1f978f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:34.550258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1b7e8803-fd28-405c-9c8f-b83bb1f978f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:34.558554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1b7e8803-fd28-405c-9c8f-b83bb1f978f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:34.560043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1b7e8803-fd28-405c-9c8f-b83bb1f978f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:34.561209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1b7e8803-fd28-405c-9c8f-b83bb1f978f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083023 Jun 10 13:22:34.561764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 913/1744] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:34 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:35.316224 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9439371e-938d-4e7b-aa11-2d1679a48636 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:35.316884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9439371e-938d-4e7b-aa11-2d1679a48636 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:35.323480 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9439371e-938d-4e7b-aa11-2d1679a48636 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:35.323922 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9439371e-938d-4e7b-aa11-2d1679a48636 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:35.324677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9439371e-938d-4e7b-aa11-2d1679a48636 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:35.325133 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9439371e-938d-4e7b-aa11-2d1679a48636 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:35.386798 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9439371e-938d-4e7b-aa11-2d1679a48636 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:35.394196 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9439371e-938d-4e7b-aa11-2d1679a48636 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082144 Jun 10 13:22:35.394749 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 832/1745] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:35 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:35.578389 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1435618d-578d-414e-84b1-13f9f4702549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:35.578910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1435618d-578d-414e-84b1-13f9f4702549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:35.585974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1435618d-578d-414e-84b1-13f9f4702549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:35.586413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1435618d-578d-414e-84b1-13f9f4702549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:35.587159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1435618d-578d-414e-84b1-13f9f4702549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:35.587581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1435618d-578d-414e-84b1-13f9f4702549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:35.649266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1435618d-578d-414e-84b1-13f9f4702549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:35.657700 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1435618d-578d-414e-84b1-13f9f4702549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:35.659807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1435618d-578d-414e-84b1-13f9f4702549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:35.661010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1435618d-578d-414e-84b1-13f9f4702549 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086667 Jun 10 13:22:35.661572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 914/1746] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:35 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:36.411488 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-26c63ca5-4e89-4c76-b75d-df0504d78bd8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:36.412094 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-26c63ca5-4e89-4c76-b75d-df0504d78bd8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:36.418675 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26c63ca5-4e89-4c76-b75d-df0504d78bd8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:36.419120 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26c63ca5-4e89-4c76-b75d-df0504d78bd8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:36.419878 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26c63ca5-4e89-4c76-b75d-df0504d78bd8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:36.420313 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-26c63ca5-4e89-4c76-b75d-df0504d78bd8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:36.483332 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-26c63ca5-4e89-4c76-b75d-df0504d78bd8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:36.491157 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-26c63ca5-4e89-4c76-b75d-df0504d78bd8 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.083689 Jun 10 13:22:36.491712 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 833/1747] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:36 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:36.679516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-41ae8d6e-30f9-4ab6-bad8-552c2fe25566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:36.680127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-41ae8d6e-30f9-4ab6-bad8-552c2fe25566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:36.686951 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-41ae8d6e-30f9-4ab6-bad8-552c2fe25566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:36.687402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-41ae8d6e-30f9-4ab6-bad8-552c2fe25566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:36.688162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-41ae8d6e-30f9-4ab6-bad8-552c2fe25566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:36.688597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-41ae8d6e-30f9-4ab6-bad8-552c2fe25566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:36.748108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-41ae8d6e-30f9-4ab6-bad8-552c2fe25566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:36.757137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-41ae8d6e-30f9-4ab6-bad8-552c2fe25566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:36.758679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-41ae8d6e-30f9-4ab6-bad8-552c2fe25566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:36.759942 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-41ae8d6e-30f9-4ab6-bad8-552c2fe25566 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084981 Jun 10 13:22:36.760566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 915/1748] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:36 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:37.507359 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-dae262f2-6724-4874-9c28-aa69b6e519c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:37.507967 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-dae262f2-6724-4874-9c28-aa69b6e519c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:37.514440 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dae262f2-6724-4874-9c28-aa69b6e519c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:37.514894 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dae262f2-6724-4874-9c28-aa69b6e519c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:37.515645 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dae262f2-6724-4874-9c28-aa69b6e519c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:37.516076 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-dae262f2-6724-4874-9c28-aa69b6e519c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:37.574405 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-dae262f2-6724-4874-9c28-aa69b6e519c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:37.581654 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-dae262f2-6724-4874-9c28-aa69b6e519c7 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.078128 Jun 10 13:22:37.582210 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 834/1749] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:37 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:37.778898 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-319cd5df-b521-410d-8ac4-758138cd3bae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:37.779507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-319cd5df-b521-410d-8ac4-758138cd3bae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:37.786227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-319cd5df-b521-410d-8ac4-758138cd3bae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:37.786672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-319cd5df-b521-410d-8ac4-758138cd3bae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:37.787429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-319cd5df-b521-410d-8ac4-758138cd3bae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:37.787858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-319cd5df-b521-410d-8ac4-758138cd3bae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:37.848796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-319cd5df-b521-410d-8ac4-758138cd3bae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:37.857332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-319cd5df-b521-410d-8ac4-758138cd3bae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:37.858809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-319cd5df-b521-410d-8ac4-758138cd3bae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:37.859935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-319cd5df-b521-410d-8ac4-758138cd3bae tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084955 Jun 10 13:22:37.860482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 916/1750] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:37 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:38.598902 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d5b3a573-6a02-4f3e-8444-29a2c9f249e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:38.599457 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d5b3a573-6a02-4f3e-8444-29a2c9f249e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:38.606225 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d5b3a573-6a02-4f3e-8444-29a2c9f249e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:38.606667 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d5b3a573-6a02-4f3e-8444-29a2c9f249e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:38.607423 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d5b3a573-6a02-4f3e-8444-29a2c9f249e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:38.607849 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d5b3a573-6a02-4f3e-8444-29a2c9f249e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:38.667159 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d5b3a573-6a02-4f3e-8444-29a2c9f249e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:38.674695 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d5b3a573-6a02-4f3e-8444-29a2c9f249e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080223 Jun 10 13:22:38.675252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 835/1751] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:38 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:38.878695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3316c039-aab0-4ef8-bb9c-f24b3c706f7e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:38.879208 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3316c039-aab0-4ef8-bb9c-f24b3c706f7e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:38.885744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3316c039-aab0-4ef8-bb9c-f24b3c706f7e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:38.886367 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3316c039-aab0-4ef8-bb9c-f24b3c706f7e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:38.886927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3316c039-aab0-4ef8-bb9c-f24b3c706f7e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:38.887361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3316c039-aab0-4ef8-bb9c-f24b3c706f7e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:38.951725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3316c039-aab0-4ef8-bb9c-f24b3c706f7e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:38.960408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3316c039-aab0-4ef8-bb9c-f24b3c706f7e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:38.961961 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3316c039-aab0-4ef8-bb9c-f24b3c706f7e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:38.963107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3316c039-aab0-4ef8-bb9c-f24b3c706f7e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.089157 Jun 10 13:22:38.963664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 917/1752] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:38 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:39.692456 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3ccacfa2-0ea9-4db5-8646-72cabddc355f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:39.693092 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3ccacfa2-0ea9-4db5-8646-72cabddc355f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:39.699246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ccacfa2-0ea9-4db5-8646-72cabddc355f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:39.699695 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ccacfa2-0ea9-4db5-8646-72cabddc355f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:39.700445 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ccacfa2-0ea9-4db5-8646-72cabddc355f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:39.700907 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ccacfa2-0ea9-4db5-8646-72cabddc355f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:39.760035 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3ccacfa2-0ea9-4db5-8646-72cabddc355f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:39.767479 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3ccacfa2-0ea9-4db5-8646-72cabddc355f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079441 Jun 10 13:22:39.768680 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 836/1753] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:39 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:39.981799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f6ab32df-09c6-4341-866c-98dba21d504f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:39.982320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f6ab32df-09c6-4341-866c-98dba21d504f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:39.988807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f6ab32df-09c6-4341-866c-98dba21d504f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:39.989248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f6ab32df-09c6-4341-866c-98dba21d504f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:39.990012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f6ab32df-09c6-4341-866c-98dba21d504f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:39.990421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f6ab32df-09c6-4341-866c-98dba21d504f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:40.052641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f6ab32df-09c6-4341-866c-98dba21d504f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:40.061048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f6ab32df-09c6-4341-866c-98dba21d504f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:40.062560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f6ab32df-09c6-4341-866c-98dba21d504f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:40.063700 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f6ab32df-09c6-4341-866c-98dba21d504f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085938 Jun 10 13:22:40.064256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 918/1754] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:39 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:40.784251 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-eb8f8b8b-6a0f-474c-9437-e1d505598f0a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:40.784908 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-eb8f8b8b-6a0f-474c-9437-e1d505598f0a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:40.791336 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eb8f8b8b-6a0f-474c-9437-e1d505598f0a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:40.791783 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eb8f8b8b-6a0f-474c-9437-e1d505598f0a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:40.792531 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eb8f8b8b-6a0f-474c-9437-e1d505598f0a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:40.793001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eb8f8b8b-6a0f-474c-9437-e1d505598f0a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:40.854629 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-eb8f8b8b-6a0f-474c-9437-e1d505598f0a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:40.862431 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-eb8f8b8b-6a0f-474c-9437-e1d505598f0a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082238 Jun 10 13:22:40.862985 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 837/1755] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:40 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:41.084248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e93294c2-8dcb-4f92-8222-be5ed7ae0335 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:41.085116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e93294c2-8dcb-4f92-8222-be5ed7ae0335 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:41.092299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e93294c2-8dcb-4f92-8222-be5ed7ae0335 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:41.092741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e93294c2-8dcb-4f92-8222-be5ed7ae0335 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:41.093543 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e93294c2-8dcb-4f92-8222-be5ed7ae0335 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:41.093969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e93294c2-8dcb-4f92-8222-be5ed7ae0335 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:41.152931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e93294c2-8dcb-4f92-8222-be5ed7ae0335 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:41.161729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e93294c2-8dcb-4f92-8222-be5ed7ae0335 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:41.163227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e93294c2-8dcb-4f92-8222-be5ed7ae0335 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:41.164356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e93294c2-8dcb-4f92-8222-be5ed7ae0335 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085617 Jun 10 13:22:41.164937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 919/1756] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:41 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:41.879566 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-49801fe7-d3d1-4023-ab25-df971f5a9a38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:41.880190 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-49801fe7-d3d1-4023-ab25-df971f5a9a38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:41.886577 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-49801fe7-d3d1-4023-ab25-df971f5a9a38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:41.887020 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-49801fe7-d3d1-4023-ab25-df971f5a9a38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:41.887768 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-49801fe7-d3d1-4023-ab25-df971f5a9a38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:41.888204 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-49801fe7-d3d1-4023-ab25-df971f5a9a38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:41.948455 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-49801fe7-d3d1-4023-ab25-df971f5a9a38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:41.956556 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-49801fe7-d3d1-4023-ab25-df971f5a9a38 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.081372 Jun 10 13:22:41.957143 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 838/1757] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:41 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:42.182184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9c73db42-acb6-4656-8e2d-1e5a0c4b889d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:42.182709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9c73db42-acb6-4656-8e2d-1e5a0c4b889d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:42.189202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c73db42-acb6-4656-8e2d-1e5a0c4b889d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:42.190155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c73db42-acb6-4656-8e2d-1e5a0c4b889d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:42.190910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c73db42-acb6-4656-8e2d-1e5a0c4b889d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:42.191341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c73db42-acb6-4656-8e2d-1e5a0c4b889d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:42.195235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:22:42.250240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9c73db42-acb6-4656-8e2d-1e5a0c4b889d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:42.259396 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9c73db42-acb6-4656-8e2d-1e5a0c4b889d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:42.260900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9c73db42-acb6-4656-8e2d-1e5a0c4b889d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:42.262039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9c73db42-acb6-4656-8e2d-1e5a0c4b889d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083789 Jun 10 13:22:42.262590 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 920/1758] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:42 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:42.974062 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-94722403-440f-4fff-a0d6-0d6e8eeb1c52 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:42.974642 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-94722403-440f-4fff-a0d6-0d6e8eeb1c52 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:42.981447 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-94722403-440f-4fff-a0d6-0d6e8eeb1c52 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:42.981881 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-94722403-440f-4fff-a0d6-0d6e8eeb1c52 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:42.982624 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-94722403-440f-4fff-a0d6-0d6e8eeb1c52 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:42.983045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-94722403-440f-4fff-a0d6-0d6e8eeb1c52 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:43.042619 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-94722403-440f-4fff-a0d6-0d6e8eeb1c52 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:43.049948 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-94722403-440f-4fff-a0d6-0d6e8eeb1c52 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080309 Jun 10 13:22:43.050515 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 839/1759] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:42 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:43.280083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-10c13c88-209c-4bda-bc91-750c8e2fb579 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:43.280617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-10c13c88-209c-4bda-bc91-750c8e2fb579 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:43.287227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10c13c88-209c-4bda-bc91-750c8e2fb579 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:43.287666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10c13c88-209c-4bda-bc91-750c8e2fb579 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:43.288414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10c13c88-209c-4bda-bc91-750c8e2fb579 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:43.288874 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10c13c88-209c-4bda-bc91-750c8e2fb579 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:43.348343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10c13c88-209c-4bda-bc91-750c8e2fb579 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:43.356941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10c13c88-209c-4bda-bc91-750c8e2fb579 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:43.358418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10c13c88-209c-4bda-bc91-750c8e2fb579 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:43.359537 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-10c13c88-209c-4bda-bc91-750c8e2fb579 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083459 Jun 10 13:22:43.360091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 921/1760] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:43 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:44.067280 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f0b83266-d1e6-4173-b904-ef9cc8de21da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:44.067888 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f0b83266-d1e6-4173-b904-ef9cc8de21da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:44.074270 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f0b83266-d1e6-4173-b904-ef9cc8de21da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:44.074713 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f0b83266-d1e6-4173-b904-ef9cc8de21da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:44.075538 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f0b83266-d1e6-4173-b904-ef9cc8de21da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:44.075970 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f0b83266-d1e6-4173-b904-ef9cc8de21da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:44.135795 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f0b83266-d1e6-4173-b904-ef9cc8de21da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:44.143271 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f0b83266-d1e6-4173-b904-ef9cc8de21da tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080465 Jun 10 13:22:44.143819 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 840/1761] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:44 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:44.377686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3c2a6215-8562-4ce6-a965-a1edd29dc454 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:44.378485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3c2a6215-8562-4ce6-a965-a1edd29dc454 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:44.380738 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=63,nova_api:UPDATE=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:22:44.385602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c2a6215-8562-4ce6-a965-a1edd29dc454 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:44.386046 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c2a6215-8562-4ce6-a965-a1edd29dc454 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:44.386799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c2a6215-8562-4ce6-a965-a1edd29dc454 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:44.387226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c2a6215-8562-4ce6-a965-a1edd29dc454 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:44.451116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3c2a6215-8562-4ce6-a965-a1edd29dc454 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:44.459485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3c2a6215-8562-4ce6-a965-a1edd29dc454 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:44.461071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3c2a6215-8562-4ce6-a965-a1edd29dc454 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:44.462969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3c2a6215-8562-4ce6-a965-a1edd29dc454 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.088857 Jun 10 13:22:44.463984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 922/1762] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:44 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:45.160481 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-fd670750-de22-431b-8292-a66dc7145142 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:45.161107 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-fd670750-de22-431b-8292-a66dc7145142 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:45.167529 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fd670750-de22-431b-8292-a66dc7145142 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:45.167971 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fd670750-de22-431b-8292-a66dc7145142 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:45.168719 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fd670750-de22-431b-8292-a66dc7145142 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:45.169171 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fd670750-de22-431b-8292-a66dc7145142 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:45.230794 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-fd670750-de22-431b-8292-a66dc7145142 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:45.238727 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-fd670750-de22-431b-8292-a66dc7145142 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082624 Jun 10 13:22:45.239849 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 841/1763] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:45 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:45.482013 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d39f3a2e-982f-4c63-814c-18c1b86cebd2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:45.482622 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d39f3a2e-982f-4c63-814c-18c1b86cebd2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:45.489239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d39f3a2e-982f-4c63-814c-18c1b86cebd2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:45.489686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d39f3a2e-982f-4c63-814c-18c1b86cebd2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:45.490444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d39f3a2e-982f-4c63-814c-18c1b86cebd2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:45.490877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d39f3a2e-982f-4c63-814c-18c1b86cebd2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:45.549176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d39f3a2e-982f-4c63-814c-18c1b86cebd2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:45.557452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d39f3a2e-982f-4c63-814c-18c1b86cebd2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:45.558948 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d39f3a2e-982f-4c63-814c-18c1b86cebd2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:45.560077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d39f3a2e-982f-4c63-814c-18c1b86cebd2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.081977 Jun 10 13:22:45.560633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 923/1764] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:45 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:46.256334 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-78c3ed4a-df62-4713-b4a2-1f50c924fa77 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:46.257474 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-78c3ed4a-df62-4713-b4a2-1f50c924fa77 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:46.263970 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-78c3ed4a-df62-4713-b4a2-1f50c924fa77 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:46.264423 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-78c3ed4a-df62-4713-b4a2-1f50c924fa77 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:46.265211 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-78c3ed4a-df62-4713-b4a2-1f50c924fa77 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:46.265646 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-78c3ed4a-df62-4713-b4a2-1f50c924fa77 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:46.577723 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-78c3ed4a-df62-4713-b4a2-1f50c924fa77 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:46.578322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-075632dc-be95-4256-9b82-f1bc5074e358 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:46.579002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-075632dc-be95-4256-9b82-f1bc5074e358 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:46.585181 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-78c3ed4a-df62-4713-b4a2-1f50c924fa77 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.333253 Jun 10 13:22:46.585594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-075632dc-be95-4256-9b82-f1bc5074e358 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:46.586008 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 842/1765] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:46 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:46.586170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-075632dc-be95-4256-9b82-f1bc5074e358 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:46.587001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-075632dc-be95-4256-9b82-f1bc5074e358 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:46.587523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-075632dc-be95-4256-9b82-f1bc5074e358 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:46.647586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-075632dc-be95-4256-9b82-f1bc5074e358 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:46.655986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-075632dc-be95-4256-9b82-f1bc5074e358 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:46.657517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-075632dc-be95-4256-9b82-f1bc5074e358 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:46.659173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-075632dc-be95-4256-9b82-f1bc5074e358 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084984 Jun 10 13:22:46.659733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 924/1766] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:46 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:22:47.602303 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5a51e583-4693-4ffa-b19b-f7a69798cfaf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:47.602847 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5a51e583-4693-4ffa-b19b-f7a69798cfaf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:47.609297 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a51e583-4693-4ffa-b19b-f7a69798cfaf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:47.609746 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a51e583-4693-4ffa-b19b-f7a69798cfaf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:47.610506 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a51e583-4693-4ffa-b19b-f7a69798cfaf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:47.610939 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5a51e583-4693-4ffa-b19b-f7a69798cfaf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:47.669714 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5a51e583-4693-4ffa-b19b-f7a69798cfaf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:47.676707 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-630ad2c6-bb87-46cd-a15e-4e40fb894c7b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:47.677204 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5a51e583-4693-4ffa-b19b-f7a69798cfaf tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079120 Jun 10 13:22:47.677537 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-630ad2c6-bb87-46cd-a15e-4e40fb894c7b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:47.677772 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 843/1767] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:47 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:47.684392 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-630ad2c6-bb87-46cd-a15e-4e40fb894c7b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:47.684875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-630ad2c6-bb87-46cd-a15e-4e40fb894c7b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:47.685676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-630ad2c6-bb87-46cd-a15e-4e40fb894c7b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:47.686114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-630ad2c6-bb87-46cd-a15e-4e40fb894c7b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:47.745034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-630ad2c6-bb87-46cd-a15e-4e40fb894c7b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:47.753264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-630ad2c6-bb87-46cd-a15e-4e40fb894c7b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:47.754779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-630ad2c6-bb87-46cd-a15e-4e40fb894c7b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:47.755915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-630ad2c6-bb87-46cd-a15e-4e40fb894c7b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083099 Jun 10 13:22:47.756466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 925/1768] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:47 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:22:48.694553 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-404859b1-716c-4b66-a188-6783997e2d3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:48.695166 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-404859b1-716c-4b66-a188-6783997e2d3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:48.697577 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=58 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:22:48.702543 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-404859b1-716c-4b66-a188-6783997e2d3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:48.702986 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-404859b1-716c-4b66-a188-6783997e2d3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:48.703743 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-404859b1-716c-4b66-a188-6783997e2d3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:48.704170 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-404859b1-716c-4b66-a188-6783997e2d3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:48.766365 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-404859b1-716c-4b66-a188-6783997e2d3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:48.774214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f0417724-1fec-474b-983f-09a1fb5335cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:48.774822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f0417724-1fec-474b-983f-09a1fb5335cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:48.775042 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-404859b1-716c-4b66-a188-6783997e2d3a tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.084248 Jun 10 13:22:48.775591 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 844/1769] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:48 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:48.781178 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0417724-1fec-474b-983f-09a1fb5335cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:48.781652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0417724-1fec-474b-983f-09a1fb5335cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:48.782418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0417724-1fec-474b-983f-09a1fb5335cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:48.782852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0417724-1fec-474b-983f-09a1fb5335cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:48.846304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f0417724-1fec-474b-983f-09a1fb5335cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:48.854972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f0417724-1fec-474b-983f-09a1fb5335cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:48.856467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f0417724-1fec-474b-983f-09a1fb5335cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:48.857647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f0417724-1fec-474b-983f-09a1fb5335cf tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087338 Jun 10 13:22:48.858198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 926/1770] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:48 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:22:49.791427 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3c8a2474-5881-46d3-a966-1cf2b62abd6b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:49.792061 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3c8a2474-5881-46d3-a966-1cf2b62abd6b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:49.798473 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c8a2474-5881-46d3-a966-1cf2b62abd6b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:49.798912 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c8a2474-5881-46d3-a966-1cf2b62abd6b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:49.799656 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c8a2474-5881-46d3-a966-1cf2b62abd6b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:49.800081 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3c8a2474-5881-46d3-a966-1cf2b62abd6b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:49.860194 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3c8a2474-5881-46d3-a966-1cf2b62abd6b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:49.867885 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3c8a2474-5881-46d3-a966-1cf2b62abd6b tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080323 Jun 10 13:22:49.868448 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 845/1771] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:49 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:49.875202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-729dcabe-6587-4e4c-b993-8ad4f119d318 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:49.876007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-729dcabe-6587-4e4c-b993-8ad4f119d318 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:49.882499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-729dcabe-6587-4e4c-b993-8ad4f119d318 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:49.882946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-729dcabe-6587-4e4c-b993-8ad4f119d318 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:49.883705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-729dcabe-6587-4e4c-b993-8ad4f119d318 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:49.884139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-729dcabe-6587-4e4c-b993-8ad4f119d318 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:49.942982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-729dcabe-6587-4e4c-b993-8ad4f119d318 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:49.951309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-729dcabe-6587-4e4c-b993-8ad4f119d318 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:49.952834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-729dcabe-6587-4e4c-b993-8ad4f119d318 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:49.953973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-729dcabe-6587-4e4c-b993-8ad4f119d318 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082608 Jun 10 13:22:49.954529 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 927/1772] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:49 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:50.884791 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b2fafc6d-b022-40e7-9420-b3f8c182dd82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:50.885417 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b2fafc6d-b022-40e7-9420-b3f8c182dd82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:50.891773 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b2fafc6d-b022-40e7-9420-b3f8c182dd82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:50.892217 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b2fafc6d-b022-40e7-9420-b3f8c182dd82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:50.893000 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b2fafc6d-b022-40e7-9420-b3f8c182dd82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:50.893426 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b2fafc6d-b022-40e7-9420-b3f8c182dd82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:50.955663 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b2fafc6d-b022-40e7-9420-b3f8c182dd82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:50.963079 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b2fafc6d-b022-40e7-9420-b3f8c182dd82 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082100 Jun 10 13:22:50.963642 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 846/1773] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:50 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:50.971841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-65ff4033-ae5d-45b2-b234-0279747ee537 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:50.972449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-65ff4033-ae5d-45b2-b234-0279747ee537 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:50.979166 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-65ff4033-ae5d-45b2-b234-0279747ee537 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:50.979616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-65ff4033-ae5d-45b2-b234-0279747ee537 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:50.980378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-65ff4033-ae5d-45b2-b234-0279747ee537 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:50.980840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-65ff4033-ae5d-45b2-b234-0279747ee537 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:51.043513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-65ff4033-ae5d-45b2-b234-0279747ee537 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:51.052385 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-65ff4033-ae5d-45b2-b234-0279747ee537 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:51.053914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-65ff4033-ae5d-45b2-b234-0279747ee537 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:51.055056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-65ff4033-ae5d-45b2-b234-0279747ee537 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087648 Jun 10 13:22:51.055613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 928/1774] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:50 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:51.981066 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ca95437f-01ee-47c1-aace-8af73cf393e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:51.981604 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ca95437f-01ee-47c1-aace-8af73cf393e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:51.988655 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ca95437f-01ee-47c1-aace-8af73cf393e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:51.989147 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ca95437f-01ee-47c1-aace-8af73cf393e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:51.989910 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ca95437f-01ee-47c1-aace-8af73cf393e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:51.990344 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ca95437f-01ee-47c1-aace-8af73cf393e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:52.007580 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:22:52.050505 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ca95437f-01ee-47c1-aace-8af73cf393e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:52.057823 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ca95437f-01ee-47c1-aace-8af73cf393e0 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080923 Jun 10 13:22:52.058386 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 847/1775] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:51 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:52.073028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8a6d845f-03d4-4a0d-b007-abfa034c3811 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:52.073628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8a6d845f-03d4-4a0d-b007-abfa034c3811 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:52.080666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a6d845f-03d4-4a0d-b007-abfa034c3811 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:52.081185 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a6d845f-03d4-4a0d-b007-abfa034c3811 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:52.081951 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a6d845f-03d4-4a0d-b007-abfa034c3811 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:52.082395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a6d845f-03d4-4a0d-b007-abfa034c3811 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:52.143524 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8a6d845f-03d4-4a0d-b007-abfa034c3811 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:52.151728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8a6d845f-03d4-4a0d-b007-abfa034c3811 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:52.153257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8a6d845f-03d4-4a0d-b007-abfa034c3811 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:52.154396 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8a6d845f-03d4-4a0d-b007-abfa034c3811 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085262 Jun 10 13:22:52.154958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 929/1776] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:52 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:53.074413 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-fc8e4748-94bb-40a5-8a2f-93351595f32e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:53.075023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-fc8e4748-94bb-40a5-8a2f-93351595f32e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:53.081508 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fc8e4748-94bb-40a5-8a2f-93351595f32e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:53.081957 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fc8e4748-94bb-40a5-8a2f-93351595f32e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:53.082720 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fc8e4748-94bb-40a5-8a2f-93351595f32e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:53.083183 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fc8e4748-94bb-40a5-8a2f-93351595f32e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:53.143964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-fc8e4748-94bb-40a5-8a2f-93351595f32e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:53.151429 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-fc8e4748-94bb-40a5-8a2f-93351595f32e tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080881 Jun 10 13:22:53.151986 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 848/1777] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:53 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:53.172775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-07240517-d12b-4ec8-87cc-c09fabbc1a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:53.174072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-07240517-d12b-4ec8-87cc-c09fabbc1a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:53.180481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07240517-d12b-4ec8-87cc-c09fabbc1a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:53.180971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07240517-d12b-4ec8-87cc-c09fabbc1a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:53.181727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07240517-d12b-4ec8-87cc-c09fabbc1a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:53.182160 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07240517-d12b-4ec8-87cc-c09fabbc1a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:53.247164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-07240517-d12b-4ec8-87cc-c09fabbc1a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:53.255736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-07240517-d12b-4ec8-87cc-c09fabbc1a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:53.257211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-07240517-d12b-4ec8-87cc-c09fabbc1a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:53.258352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-07240517-d12b-4ec8-87cc-c09fabbc1a49 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.089509 Jun 10 13:22:53.258901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 930/1778] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:53 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:54.168216 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7c5156bf-67b2-45dd-b712-1aa8aafa3d2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:54.168853 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7c5156bf-67b2-45dd-b712-1aa8aafa3d2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:54.175258 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7c5156bf-67b2-45dd-b712-1aa8aafa3d2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:54.175703 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7c5156bf-67b2-45dd-b712-1aa8aafa3d2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:54.176458 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7c5156bf-67b2-45dd-b712-1aa8aafa3d2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:54.176932 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7c5156bf-67b2-45dd-b712-1aa8aafa3d2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:54.235729 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7c5156bf-67b2-45dd-b712-1aa8aafa3d2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:54.243121 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7c5156bf-67b2-45dd-b712-1aa8aafa3d2f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.078728 Jun 10 13:22:54.243680 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 849/1779] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:54 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:54.275252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7d56cceb-4e92-490e-849f-91d298470ed7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:54.275851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7d56cceb-4e92-490e-849f-91d298470ed7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:54.282549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d56cceb-4e92-490e-849f-91d298470ed7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:54.282750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d56cceb-4e92-490e-849f-91d298470ed7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:54.283499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d56cceb-4e92-490e-849f-91d298470ed7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:54.283929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d56cceb-4e92-490e-849f-91d298470ed7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:54.343388 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7d56cceb-4e92-490e-849f-91d298470ed7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:54.352345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7d56cceb-4e92-490e-849f-91d298470ed7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:54.353876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7d56cceb-4e92-490e-849f-91d298470ed7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:54.355003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7d56cceb-4e92-490e-849f-91d298470ed7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083629 Jun 10 13:22:54.355557 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 931/1780] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:54 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:55.260166 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0f0eec8a-3341-4dda-add2-06d2930ca951 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:55.260822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0f0eec8a-3341-4dda-add2-06d2930ca951 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:55.267825 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f0eec8a-3341-4dda-add2-06d2930ca951 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:55.268274 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f0eec8a-3341-4dda-add2-06d2930ca951 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:55.269060 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f0eec8a-3341-4dda-add2-06d2930ca951 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:55.269495 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0f0eec8a-3341-4dda-add2-06d2930ca951 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:55.331647 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0f0eec8a-3341-4dda-add2-06d2930ca951 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:55.339146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0f0eec8a-3341-4dda-add2-06d2930ca951 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.082805 Jun 10 13:22:55.339708 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 850/1781] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:55 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:55.373225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-66bbe559-3fa7-47a3-8b5e-ebcd8d07dd35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:55.373863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-66bbe559-3fa7-47a3-8b5e-ebcd8d07dd35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:55.380900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-66bbe559-3fa7-47a3-8b5e-ebcd8d07dd35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:55.381353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-66bbe559-3fa7-47a3-8b5e-ebcd8d07dd35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:55.382116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-66bbe559-3fa7-47a3-8b5e-ebcd8d07dd35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:55.382554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-66bbe559-3fa7-47a3-8b5e-ebcd8d07dd35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:55.441187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-66bbe559-3fa7-47a3-8b5e-ebcd8d07dd35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:55.452747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-66bbe559-3fa7-47a3-8b5e-ebcd8d07dd35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:55.455811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-66bbe559-3fa7-47a3-8b5e-ebcd8d07dd35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:55.457588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-66bbe559-3fa7-47a3-8b5e-ebcd8d07dd35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.088091 Jun 10 13:22:55.458409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 932/1782] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:55 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:56.356589 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-b416dbae-5d53-4830-b033-fc49ed58bde2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:56.357223 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-b416dbae-5d53-4830-b033-fc49ed58bde2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:56.364256 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b416dbae-5d53-4830-b033-fc49ed58bde2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:56.364705 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b416dbae-5d53-4830-b033-fc49ed58bde2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:56.365477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b416dbae-5d53-4830-b033-fc49ed58bde2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:56.365903 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-b416dbae-5d53-4830-b033-fc49ed58bde2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:56.425952 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-b416dbae-5d53-4830-b033-fc49ed58bde2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:56.433133 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-b416dbae-5d53-4830-b033-fc49ed58bde2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080388 Jun 10 13:22:56.433688 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 851/1783] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:56 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:56.481338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4aee70dc-56d1-4ccf-892f-be13765f7777 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:56.481982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4aee70dc-56d1-4ccf-892f-be13765f7777 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:56.489205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4aee70dc-56d1-4ccf-892f-be13765f7777 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:56.489658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4aee70dc-56d1-4ccf-892f-be13765f7777 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:56.490418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4aee70dc-56d1-4ccf-892f-be13765f7777 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:56.490856 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4aee70dc-56d1-4ccf-892f-be13765f7777 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:56.554030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4aee70dc-56d1-4ccf-892f-be13765f7777 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:56.562267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4aee70dc-56d1-4ccf-892f-be13765f7777 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:56.563771 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4aee70dc-56d1-4ccf-892f-be13765f7777 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:56.565038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4aee70dc-56d1-4ccf-892f-be13765f7777 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087527 Jun 10 13:22:56.565593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 933/1784] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:56 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:57.449865 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-411170ea-464d-456c-bf32-efb76855e108 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:57.450477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-411170ea-464d-456c-bf32-efb76855e108 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:57.457001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-411170ea-464d-456c-bf32-efb76855e108 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:57.457449 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-411170ea-464d-456c-bf32-efb76855e108 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:57.458212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-411170ea-464d-456c-bf32-efb76855e108 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:57.458644 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-411170ea-464d-456c-bf32-efb76855e108 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:57.521467 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-411170ea-464d-456c-bf32-efb76855e108 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:57.532363 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-411170ea-464d-456c-bf32-efb76855e108 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.086325 Jun 10 13:22:57.533030 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 852/1785] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:57 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:57.582279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-68a6ba83-5bf9-4da7-a2f3-4fd3094bf31b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:57.582918 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-68a6ba83-5bf9-4da7-a2f3-4fd3094bf31b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:57.589427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68a6ba83-5bf9-4da7-a2f3-4fd3094bf31b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:57.589878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68a6ba83-5bf9-4da7-a2f3-4fd3094bf31b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:57.590641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68a6ba83-5bf9-4da7-a2f3-4fd3094bf31b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:57.591080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68a6ba83-5bf9-4da7-a2f3-4fd3094bf31b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:57.652794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-68a6ba83-5bf9-4da7-a2f3-4fd3094bf31b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:57.661599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-68a6ba83-5bf9-4da7-a2f3-4fd3094bf31b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:57.663103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-68a6ba83-5bf9-4da7-a2f3-4fd3094bf31b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:57.664273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-68a6ba83-5bf9-4da7-a2f3-4fd3094bf31b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085864 Jun 10 13:22:57.664816 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 934/1786] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:57 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:58.549966 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a339f728-8168-49d3-9cd3-a2402f3fc286 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:58.550523 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a339f728-8168-49d3-9cd3-a2402f3fc286 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:58.556882 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a339f728-8168-49d3-9cd3-a2402f3fc286 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:58.557321 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a339f728-8168-49d3-9cd3-a2402f3fc286 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:58.558066 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a339f728-8168-49d3-9cd3-a2402f3fc286 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:58.558495 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a339f728-8168-49d3-9cd3-a2402f3fc286 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:58.617322 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a339f728-8168-49d3-9cd3-a2402f3fc286 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:58.624592 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a339f728-8168-49d3-9cd3-a2402f3fc286 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.078516 Jun 10 13:22:58.625180 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 853/1787] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:58 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:58.682599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3f6a07ee-c1f4-4930-835d-0f36285f70fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:58.683212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3f6a07ee-c1f4-4930-835d-0f36285f70fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:58.690216 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f6a07ee-c1f4-4930-835d-0f36285f70fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:58.690659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f6a07ee-c1f4-4930-835d-0f36285f70fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:58.691408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f6a07ee-c1f4-4930-835d-0f36285f70fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:58.691839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f6a07ee-c1f4-4930-835d-0f36285f70fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:58.754096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f6a07ee-c1f4-4930-835d-0f36285f70fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:58.762555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f6a07ee-c1f4-4930-835d-0f36285f70fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:58.764055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f6a07ee-c1f4-4930-835d-0f36285f70fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:58.765236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3f6a07ee-c1f4-4930-835d-0f36285f70fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086501 Jun 10 13:22:58.765804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 935/1788] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:58 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:22:59.642019 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9c8dabc7-2c42-4f29-9e05-74b1768546c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:59.642557 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9c8dabc7-2c42-4f29-9e05-74b1768546c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:59.649691 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9c8dabc7-2c42-4f29-9e05-74b1768546c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:59.650147 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9c8dabc7-2c42-4f29-9e05-74b1768546c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:59.650910 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9c8dabc7-2c42-4f29-9e05-74b1768546c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:59.651346 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9c8dabc7-2c42-4f29-9e05-74b1768546c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:59.714374 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9c8dabc7-2c42-4f29-9e05-74b1768546c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:59.722432 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9c8dabc7-2c42-4f29-9e05-74b1768546c2 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.084422 Jun 10 13:22:59.722995 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 854/1789] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:22:59 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:22:59.782991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2a2753d0-6874-4e9b-ab4d-5385298fa918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:22:59.784220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2a2753d0-6874-4e9b-ab4d-5385298fa918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:22:59.790658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2a2753d0-6874-4e9b-ab4d-5385298fa918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:59.791107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2a2753d0-6874-4e9b-ab4d-5385298fa918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:59.791858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2a2753d0-6874-4e9b-ab4d-5385298fa918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:22:59.792293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2a2753d0-6874-4e9b-ab4d-5385298fa918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:22:59.852535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2a2753d0-6874-4e9b-ab4d-5385298fa918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:59.855902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:22:59.861067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2a2753d0-6874-4e9b-ab4d-5385298fa918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:59.862544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2a2753d0-6874-4e9b-ab4d-5385298fa918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:22:59.863674 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2a2753d0-6874-4e9b-ab4d-5385298fa918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084614 Jun 10 13:22:59.864220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 936/1790] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:22:59 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:00.740182 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-875e8f5c-44e4-492a-b2ca-37ddf8c09bff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:00.740811 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-875e8f5c-44e4-492a-b2ca-37ddf8c09bff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:00.747287 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-875e8f5c-44e4-492a-b2ca-37ddf8c09bff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:00.747714 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-875e8f5c-44e4-492a-b2ca-37ddf8c09bff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:00.748466 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-875e8f5c-44e4-492a-b2ca-37ddf8c09bff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:00.748934 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-875e8f5c-44e4-492a-b2ca-37ddf8c09bff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:00.807422 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-875e8f5c-44e4-492a-b2ca-37ddf8c09bff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:00.814658 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-875e8f5c-44e4-492a-b2ca-37ddf8c09bff tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079000 Jun 10 13:23:00.815213 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 855/1791] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:23:00 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:23:00.881680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ad193d7a-3e48-47ce-977d-e4038b3b237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:00.882286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ad193d7a-3e48-47ce-977d-e4038b3b237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:00.888744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad193d7a-3e48-47ce-977d-e4038b3b237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:00.889218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad193d7a-3e48-47ce-977d-e4038b3b237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:00.889977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad193d7a-3e48-47ce-977d-e4038b3b237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:00.890413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad193d7a-3e48-47ce-977d-e4038b3b237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:00.952555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ad193d7a-3e48-47ce-977d-e4038b3b237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:00.960950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ad193d7a-3e48-47ce-977d-e4038b3b237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:00.962437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ad193d7a-3e48-47ce-977d-e4038b3b237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:00.963573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ad193d7a-3e48-47ce-977d-e4038b3b237f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086112 Jun 10 13:23:00.964131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 937/1792] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:00 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:01.831889 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-fcb294d0-8dbf-4bb1-a25c-356531d52ff6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:01.832496 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-fcb294d0-8dbf-4bb1-a25c-356531d52ff6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:01.838991 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fcb294d0-8dbf-4bb1-a25c-356531d52ff6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:01.839436 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fcb294d0-8dbf-4bb1-a25c-356531d52ff6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:01.840197 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fcb294d0-8dbf-4bb1-a25c-356531d52ff6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:01.840628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fcb294d0-8dbf-4bb1-a25c-356531d52ff6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:01.899888 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-fcb294d0-8dbf-4bb1-a25c-356531d52ff6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:01.907277 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-fcb294d0-8dbf-4bb1-a25c-356531d52ff6 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079734 Jun 10 13:23:01.907835 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 856/1793] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:23:01 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:23:01.981012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2ce65712-7022-4cb6-9c4c-9eee31c98040 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:01.981615 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2ce65712-7022-4cb6-9c4c-9eee31c98040 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:01.987551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ce65712-7022-4cb6-9c4c-9eee31c98040 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:01.987989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ce65712-7022-4cb6-9c4c-9eee31c98040 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:01.988737 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ce65712-7022-4cb6-9c4c-9eee31c98040 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:01.989188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ce65712-7022-4cb6-9c4c-9eee31c98040 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:02.050787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2ce65712-7022-4cb6-9c4c-9eee31c98040 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:02.058902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2ce65712-7022-4cb6-9c4c-9eee31c98040 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:02.060426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2ce65712-7022-4cb6-9c4c-9eee31c98040 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:02.061569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2ce65712-7022-4cb6-9c4c-9eee31c98040 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084570 Jun 10 13:23:02.062119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 938/1794] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:01 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:02.925010 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-fada3827-bf5d-46c0-8f8a-407b24a6fe3f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:02.925586 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-fada3827-bf5d-46c0-8f8a-407b24a6fe3f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:02.932075 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fada3827-bf5d-46c0-8f8a-407b24a6fe3f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:02.932520 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fada3827-bf5d-46c0-8f8a-407b24a6fe3f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:02.933303 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fada3827-bf5d-46c0-8f8a-407b24a6fe3f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:02.933737 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-fada3827-bf5d-46c0-8f8a-407b24a6fe3f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:02.994682 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-fada3827-bf5d-46c0-8f8a-407b24a6fe3f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:03.002023 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-fada3827-bf5d-46c0-8f8a-407b24a6fe3f tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.081579 Jun 10 13:23:03.002579 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 857/1795] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:23:02 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:23:03.078919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3a258aed-c276-4fd4-a8ae-1eb48939695b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:03.079507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3a258aed-c276-4fd4-a8ae-1eb48939695b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:03.085639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3a258aed-c276-4fd4-a8ae-1eb48939695b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:03.086583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3a258aed-c276-4fd4-a8ae-1eb48939695b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:03.087005 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3a258aed-c276-4fd4-a8ae-1eb48939695b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:03.087444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3a258aed-c276-4fd4-a8ae-1eb48939695b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:03.147254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3a258aed-c276-4fd4-a8ae-1eb48939695b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:03.155565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3a258aed-c276-4fd4-a8ae-1eb48939695b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:03.157052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3a258aed-c276-4fd4-a8ae-1eb48939695b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:03.158721 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3a258aed-c276-4fd4-a8ae-1eb48939695b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083210 Jun 10 13:23:03.159278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 939/1796] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:03 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:04.017875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-0753d609-cb9f-4b49-9d87-b8350af42246 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:04.018480 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-0753d609-cb9f-4b49-9d87-b8350af42246 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:04.024924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0753d609-cb9f-4b49-9d87-b8350af42246 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:04.025369 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0753d609-cb9f-4b49-9d87-b8350af42246 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:04.026115 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0753d609-cb9f-4b49-9d87-b8350af42246 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:04.026543 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-0753d609-cb9f-4b49-9d87-b8350af42246 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:04.093535 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-0753d609-cb9f-4b49-9d87-b8350af42246 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:04.095268 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-0753d609-cb9f-4b49-9d87-b8350af42246 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.081239 Jun 10 13:23:04.095819 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 858/1797] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:23:04 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:23:04.176883 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1067471c-9802-4b87-81a3-ba5de241f725 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:04.177489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1067471c-9802-4b87-81a3-ba5de241f725 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:04.184033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1067471c-9802-4b87-81a3-ba5de241f725 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:04.184508 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1067471c-9802-4b87-81a3-ba5de241f725 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:04.185304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1067471c-9802-4b87-81a3-ba5de241f725 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:04.185733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1067471c-9802-4b87-81a3-ba5de241f725 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:04.245163 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1067471c-9802-4b87-81a3-ba5de241f725 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:04.258400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1067471c-9802-4b87-81a3-ba5de241f725 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:04.259877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1067471c-9802-4b87-81a3-ba5de241f725 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:04.261048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1067471c-9802-4b87-81a3-ba5de241f725 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.088568 Jun 10 13:23:04.261603 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 940/1798] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:04 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:04.362511 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:23:04.362684 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:23:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:23:05.112995 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4bed659e-c775-4b26-8ce8-0a87d4f5acae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:05.113605 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4bed659e-c775-4b26-8ce8-0a87d4f5acae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:05.120055 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4bed659e-c775-4b26-8ce8-0a87d4f5acae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:05.120501 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4bed659e-c775-4b26-8ce8-0a87d4f5acae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:05.121291 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4bed659e-c775-4b26-8ce8-0a87d4f5acae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:05.121724 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4bed659e-c775-4b26-8ce8-0a87d4f5acae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:05.181231 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4bed659e-c775-4b26-8ce8-0a87d4f5acae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:05.188525 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4bed659e-c775-4b26-8ce8-0a87d4f5acae tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.079919 Jun 10 13:23:05.189114 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 859/1799] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:23:05 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:23:05.278673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9eeb34bb-e09d-4eb5-a8da-33e302dd3568 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:05.279321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9eeb34bb-e09d-4eb5-a8da-33e302dd3568 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:05.286225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9eeb34bb-e09d-4eb5-a8da-33e302dd3568 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:05.286676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9eeb34bb-e09d-4eb5-a8da-33e302dd3568 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:05.287410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9eeb34bb-e09d-4eb5-a8da-33e302dd3568 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:05.287842 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9eeb34bb-e09d-4eb5-a8da-33e302dd3568 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:05.347760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9eeb34bb-e09d-4eb5-a8da-33e302dd3568 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:05.356370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9eeb34bb-e09d-4eb5-a8da-33e302dd3568 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:05.357886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9eeb34bb-e09d-4eb5-a8da-33e302dd3568 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:05.359025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9eeb34bb-e09d-4eb5-a8da-33e302dd3568 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084204 Jun 10 13:23:05.359571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 941/1800] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:05 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:06.206064 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-04822f77-421c-47d9-bf13-3065d025c2d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:06.206695 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-04822f77-421c-47d9-bf13-3065d025c2d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] [instance: 3baaefc5-1a36-4fa6-88d9-1764e5631a32] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:06.213318 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04822f77-421c-47d9-bf13-3065d025c2d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:06.213764 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04822f77-421c-47d9-bf13-3065d025c2d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:06.214527 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04822f77-421c-47d9-bf13-3065d025c2d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:06.214959 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-04822f77-421c-47d9-bf13-3065d025c2d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:06.274984 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-04822f77-421c-47d9-bf13-3065d025c2d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0a1368d42efc4543a908a61c48ec4472', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75273f09637240f3a5cc7129927765f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:06.282442 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-04822f77-421c-47d9-bf13-3065d025c2d9 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32" status: 200 len: 1213 microversion: 2.1 time: 0.080864 Jun 10 13:23:06.283646 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 860/1801] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:23:06 2022] GET /compute/v2.1/servers/3baaefc5-1a36-4fa6-88d9-1764e5631a32 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:23:06.378153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a6780193-640c-4b96-ae3e-b3d7b0ff4328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:06.378863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a6780193-640c-4b96-ae3e-b3d7b0ff4328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:06.385330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a6780193-640c-4b96-ae3e-b3d7b0ff4328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:06.385779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a6780193-640c-4b96-ae3e-b3d7b0ff4328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:06.386542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a6780193-640c-4b96-ae3e-b3d7b0ff4328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:06.386975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a6780193-640c-4b96-ae3e-b3d7b0ff4328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:06.449818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a6780193-640c-4b96-ae3e-b3d7b0ff4328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:06.458405 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a6780193-640c-4b96-ae3e-b3d7b0ff4328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:06.459829 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a6780193-640c-4b96-ae3e-b3d7b0ff4328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:06.460984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a6780193-640c-4b96-ae3e-b3d7b0ff4328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087386 Jun 10 13:23:06.461530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 942/1802] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:06 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:06.511423 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a6c2602b-c904-441c-a096-7db698c70cf5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] Calling method '.version_select at 0x7f93052145e0>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:06.565542 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a6c2602b-c904-441c-a096-7db698c70cf5 tempest-TestShelveInstance-1739324634 tempest-TestShelveInstance-1739324634-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-943261541" status: 202 len: 0 microversion: 2.1 time: 0.057890 Jun 10 13:23:06.568394 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 861/1803] 10.222.0.25 () {58 vars in 1250 bytes} [Fri Jun 10 13:23:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-943261541 => generated 0 bytes in 59 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jun 10 13:23:07.479227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-47c09928-5af9-41af-a3ff-1fda2b927368 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:07.479888 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-47c09928-5af9-41af-a3ff-1fda2b927368 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:07.486517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-47c09928-5af9-41af-a3ff-1fda2b927368 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:07.486964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-47c09928-5af9-41af-a3ff-1fda2b927368 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:07.487814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-47c09928-5af9-41af-a3ff-1fda2b927368 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:07.488249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-47c09928-5af9-41af-a3ff-1fda2b927368 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:07.549491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-47c09928-5af9-41af-a3ff-1fda2b927368 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:07.558185 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-47c09928-5af9-41af-a3ff-1fda2b927368 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:07.560624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-47c09928-5af9-41af-a3ff-1fda2b927368 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:07.560908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-47c09928-5af9-41af-a3ff-1fda2b927368 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085542 Jun 10 13:23:07.561474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 943/1804] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:07 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:08.578972 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-12be40c0-117a-45bb-a305-0fff1c449703 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:08.579661 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-12be40c0-117a-45bb-a305-0fff1c449703 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:08.586266 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12be40c0-117a-45bb-a305-0fff1c449703 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:08.586920 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12be40c0-117a-45bb-a305-0fff1c449703 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:08.587685 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12be40c0-117a-45bb-a305-0fff1c449703 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:08.588109 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12be40c0-117a-45bb-a305-0fff1c449703 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:08.659309 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-12be40c0-117a-45bb-a305-0fff1c449703 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:08.668321 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-12be40c0-117a-45bb-a305-0fff1c449703 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:08.670449 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-12be40c0-117a-45bb-a305-0fff1c449703 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:08.671611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-12be40c0-117a-45bb-a305-0fff1c449703 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.097115 Jun 10 13:23:08.672158 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 862/1805] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:08 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:09.689900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-55b17d30-0d36-4dbd-91dd-75bcbd3abd63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:09.691258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-55b17d30-0d36-4dbd-91dd-75bcbd3abd63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:09.697949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55b17d30-0d36-4dbd-91dd-75bcbd3abd63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:09.698678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55b17d30-0d36-4dbd-91dd-75bcbd3abd63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:09.699460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55b17d30-0d36-4dbd-91dd-75bcbd3abd63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:09.700059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55b17d30-0d36-4dbd-91dd-75bcbd3abd63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:09.763755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-55b17d30-0d36-4dbd-91dd-75bcbd3abd63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:09.772290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-55b17d30-0d36-4dbd-91dd-75bcbd3abd63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:09.773813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-55b17d30-0d36-4dbd-91dd-75bcbd3abd63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:09.776328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-55b17d30-0d36-4dbd-91dd-75bcbd3abd63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.089782 Jun 10 13:23:09.776328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 944/1806] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:09 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:10.794298 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-12c0b37d-847e-4aac-a3a0-f1842bf69ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:10.795013 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-12c0b37d-847e-4aac-a3a0-f1842bf69ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:10.802083 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12c0b37d-847e-4aac-a3a0-f1842bf69ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:10.802519 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12c0b37d-847e-4aac-a3a0-f1842bf69ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:10.803274 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12c0b37d-847e-4aac-a3a0-f1842bf69ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:10.803698 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-12c0b37d-847e-4aac-a3a0-f1842bf69ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:10.865701 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-12c0b37d-847e-4aac-a3a0-f1842bf69ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:10.874206 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-12c0b37d-847e-4aac-a3a0-f1842bf69ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:10.875709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-12c0b37d-847e-4aac-a3a0-f1842bf69ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:10.877465 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-12c0b37d-847e-4aac-a3a0-f1842bf69ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086393 Jun 10 13:23:10.878021 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 863/1807] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:10 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:23:11.896124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8abae405-a1e2-4e2d-a333-9e49eb20212e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:11.896672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8abae405-a1e2-4e2d-a333-9e49eb20212e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:11.903196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8abae405-a1e2-4e2d-a333-9e49eb20212e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:11.903647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8abae405-a1e2-4e2d-a333-9e49eb20212e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:11.904422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8abae405-a1e2-4e2d-a333-9e49eb20212e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:11.904883 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8abae405-a1e2-4e2d-a333-9e49eb20212e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:12.008635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8abae405-a1e2-4e2d-a333-9e49eb20212e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:12.017094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8abae405-a1e2-4e2d-a333-9e49eb20212e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:12.018591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8abae405-a1e2-4e2d-a333-9e49eb20212e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:12.019732 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8abae405-a1e2-4e2d-a333-9e49eb20212e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.128242 Jun 10 13:23:12.020488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 945/1808] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:11 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:13.038136 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a655db2f-6fd1-48ff-82b4-ff7e63ebe5d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:13.038705 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a655db2f-6fd1-48ff-82b4-ff7e63ebe5d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:13.045873 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a655db2f-6fd1-48ff-82b4-ff7e63ebe5d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:13.046430 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a655db2f-6fd1-48ff-82b4-ff7e63ebe5d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:13.047252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a655db2f-6fd1-48ff-82b4-ff7e63ebe5d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:13.047691 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a655db2f-6fd1-48ff-82b4-ff7e63ebe5d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:13.122061 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a655db2f-6fd1-48ff-82b4-ff7e63ebe5d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:13.131418 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a655db2f-6fd1-48ff-82b4-ff7e63ebe5d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:13.133197 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a655db2f-6fd1-48ff-82b4-ff7e63ebe5d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:13.134456 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a655db2f-6fd1-48ff-82b4-ff7e63ebe5d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.100543 Jun 10 13:23:13.135082 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 864/1809] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:13 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:14.152375 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a459e17b-ceb6-4914-8b6c-674c2a8d9326 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:14.153070 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a459e17b-ceb6-4914-8b6c-674c2a8d9326 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:14.160139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a459e17b-ceb6-4914-8b6c-674c2a8d9326 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:14.160624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a459e17b-ceb6-4914-8b6c-674c2a8d9326 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:14.161557 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a459e17b-ceb6-4914-8b6c-674c2a8d9326 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:14.161998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a459e17b-ceb6-4914-8b6c-674c2a8d9326 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:14.229852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a459e17b-ceb6-4914-8b6c-674c2a8d9326 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:14.239928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a459e17b-ceb6-4914-8b6c-674c2a8d9326 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:14.241677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a459e17b-ceb6-4914-8b6c-674c2a8d9326 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:14.242875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a459e17b-ceb6-4914-8b6c-674c2a8d9326 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.094612 Jun 10 13:23:14.243419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 946/1810] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:14 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:15.260364 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6f379131-c274-4347-a8c8-ca781e5798df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:15.261086 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6f379131-c274-4347-a8c8-ca781e5798df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:15.268177 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6f379131-c274-4347-a8c8-ca781e5798df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:15.268621 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6f379131-c274-4347-a8c8-ca781e5798df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:15.271091 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6f379131-c274-4347-a8c8-ca781e5798df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:15.271091 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6f379131-c274-4347-a8c8-ca781e5798df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:15.273973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=100 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:23:15.338822 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6f379131-c274-4347-a8c8-ca781e5798df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:15.347310 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6f379131-c274-4347-a8c8-ca781e5798df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:15.351532 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6f379131-c274-4347-a8c8-ca781e5798df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:15.351532 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6f379131-c274-4347-a8c8-ca781e5798df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.093620 Jun 10 13:23:15.351532 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 865/1811] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:15 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:16.367722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cad5a049-5294-485c-bb9e-a2deac0818f4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:16.368323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cad5a049-5294-485c-bb9e-a2deac0818f4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:16.370659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=50 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:23:16.375344 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cad5a049-5294-485c-bb9e-a2deac0818f4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:16.375787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cad5a049-5294-485c-bb9e-a2deac0818f4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:16.377197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cad5a049-5294-485c-bb9e-a2deac0818f4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:16.377628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cad5a049-5294-485c-bb9e-a2deac0818f4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:16.440979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cad5a049-5294-485c-bb9e-a2deac0818f4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:16.449157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cad5a049-5294-485c-bb9e-a2deac0818f4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:16.450650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cad5a049-5294-485c-bb9e-a2deac0818f4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:16.451774 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cad5a049-5294-485c-bb9e-a2deac0818f4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087972 Jun 10 13:23:16.452321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 947/1812] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:16 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:17.469529 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-70545e88-07ce-451f-a22b-ede4497e0965 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:17.470078 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-70545e88-07ce-451f-a22b-ede4497e0965 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:17.477176 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-70545e88-07ce-451f-a22b-ede4497e0965 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:17.477614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-70545e88-07ce-451f-a22b-ede4497e0965 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:17.478369 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-70545e88-07ce-451f-a22b-ede4497e0965 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:17.478796 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-70545e88-07ce-451f-a22b-ede4497e0965 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:17.537326 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-70545e88-07ce-451f-a22b-ede4497e0965 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:17.545475 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-70545e88-07ce-451f-a22b-ede4497e0965 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:17.546949 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-70545e88-07ce-451f-a22b-ede4497e0965 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:17.548570 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-70545e88-07ce-451f-a22b-ede4497e0965 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082488 Jun 10 13:23:17.549150 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 866/1813] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:17 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:23:18.567494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-836ffe26-19d4-43d7-8b5b-9260dad63300 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:18.568038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-836ffe26-19d4-43d7-8b5b-9260dad63300 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:18.577784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-836ffe26-19d4-43d7-8b5b-9260dad63300 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:18.578240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-836ffe26-19d4-43d7-8b5b-9260dad63300 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:18.579017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-836ffe26-19d4-43d7-8b5b-9260dad63300 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:18.579496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-836ffe26-19d4-43d7-8b5b-9260dad63300 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:18.639578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-836ffe26-19d4-43d7-8b5b-9260dad63300 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:18.648120 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-836ffe26-19d4-43d7-8b5b-9260dad63300 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:18.649639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-836ffe26-19d4-43d7-8b5b-9260dad63300 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:18.650804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-836ffe26-19d4-43d7-8b5b-9260dad63300 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087958 Jun 10 13:23:18.651358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 948/1814] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:18 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:19.668231 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2b46a9d8-32e3-46f3-97e4-e30c6fe393c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:19.668793 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2b46a9d8-32e3-46f3-97e4-e30c6fe393c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:19.671166 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=47,nova_api:DELETE=1 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:23:19.675889 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2b46a9d8-32e3-46f3-97e4-e30c6fe393c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:19.676337 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2b46a9d8-32e3-46f3-97e4-e30c6fe393c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:19.677131 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2b46a9d8-32e3-46f3-97e4-e30c6fe393c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:19.677566 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2b46a9d8-32e3-46f3-97e4-e30c6fe393c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:19.742217 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2b46a9d8-32e3-46f3-97e4-e30c6fe393c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:19.750455 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2b46a9d8-32e3-46f3-97e4-e30c6fe393c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:19.752610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2b46a9d8-32e3-46f3-97e4-e30c6fe393c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:19.753775 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2b46a9d8-32e3-46f3-97e4-e30c6fe393c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.089510 Jun 10 13:23:19.754332 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 867/1815] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:19 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:20.772185 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e8640582-2cce-4828-b7b2-d930105345eb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:20.772862 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e8640582-2cce-4828-b7b2-d930105345eb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:20.779521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8640582-2cce-4828-b7b2-d930105345eb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:20.779966 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8640582-2cce-4828-b7b2-d930105345eb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:20.780729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8640582-2cce-4828-b7b2-d930105345eb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:20.781182 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8640582-2cce-4828-b7b2-d930105345eb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:20.848323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e8640582-2cce-4828-b7b2-d930105345eb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:20.857403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e8640582-2cce-4828-b7b2-d930105345eb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:20.858875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e8640582-2cce-4828-b7b2-d930105345eb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:20.860007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e8640582-2cce-4828-b7b2-d930105345eb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.092539 Jun 10 13:23:20.860595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 949/1816] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:20 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:21.877977 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ed320e85-30cc-48c8-8a87-bfee415cb064 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:21.878591 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ed320e85-30cc-48c8-8a87-bfee415cb064 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:21.885450 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ed320e85-30cc-48c8-8a87-bfee415cb064 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:21.885897 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ed320e85-30cc-48c8-8a87-bfee415cb064 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:21.886663 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ed320e85-30cc-48c8-8a87-bfee415cb064 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:21.887127 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ed320e85-30cc-48c8-8a87-bfee415cb064 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:21.947153 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ed320e85-30cc-48c8-8a87-bfee415cb064 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:21.955997 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ed320e85-30cc-48c8-8a87-bfee415cb064 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:21.957517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ed320e85-30cc-48c8-8a87-bfee415cb064 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:21.958739 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ed320e85-30cc-48c8-8a87-bfee415cb064 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084719 Jun 10 13:23:21.959289 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 868/1817] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:21 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:23:22.977296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a61119c2-3fe2-4b66-b351-17638af0163c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:22.977898 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a61119c2-3fe2-4b66-b351-17638af0163c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:22.984337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a61119c2-3fe2-4b66-b351-17638af0163c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:22.984801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a61119c2-3fe2-4b66-b351-17638af0163c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:22.985583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a61119c2-3fe2-4b66-b351-17638af0163c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:22.986010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a61119c2-3fe2-4b66-b351-17638af0163c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:23.044381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a61119c2-3fe2-4b66-b351-17638af0163c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:23.053745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a61119c2-3fe2-4b66-b351-17638af0163c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:23.055491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a61119c2-3fe2-4b66-b351-17638af0163c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:23.056617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a61119c2-3fe2-4b66-b351-17638af0163c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083498 Jun 10 13:23:23.057192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 950/1818] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:22 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:24.074220 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-406806a2-7f61-470c-b889-6898965248da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:24.074817 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-406806a2-7f61-470c-b889-6898965248da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:24.081539 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-406806a2-7f61-470c-b889-6898965248da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:24.081993 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-406806a2-7f61-470c-b889-6898965248da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:24.082762 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-406806a2-7f61-470c-b889-6898965248da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:24.083201 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-406806a2-7f61-470c-b889-6898965248da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:24.143973 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-406806a2-7f61-470c-b889-6898965248da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:24.152265 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-406806a2-7f61-470c-b889-6898965248da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:24.153777 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-406806a2-7f61-470c-b889-6898965248da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:24.154907 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-406806a2-7f61-470c-b889-6898965248da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084639 Jun 10 13:23:24.155459 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 869/1819] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:24 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:25.172644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-39754970-680e-4d8e-bb99-20cc162be9e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:25.173316 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-39754970-680e-4d8e-bb99-20cc162be9e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:25.180373 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39754970-680e-4d8e-bb99-20cc162be9e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:25.180838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39754970-680e-4d8e-bb99-20cc162be9e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:25.181593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39754970-680e-4d8e-bb99-20cc162be9e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:25.182019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39754970-680e-4d8e-bb99-20cc162be9e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:25.244738 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-39754970-680e-4d8e-bb99-20cc162be9e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:25.253202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-39754970-680e-4d8e-bb99-20cc162be9e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:25.254718 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-39754970-680e-4d8e-bb99-20cc162be9e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:25.255859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-39754970-680e-4d8e-bb99-20cc162be9e4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087170 Jun 10 13:23:25.256412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 951/1820] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:25 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:26.274488 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cc7eee58-050d-4956-bbc9-b703af76e10b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:26.275042 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cc7eee58-050d-4956-bbc9-b703af76e10b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:26.281509 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc7eee58-050d-4956-bbc9-b703af76e10b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:26.281954 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc7eee58-050d-4956-bbc9-b703af76e10b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:26.282707 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc7eee58-050d-4956-bbc9-b703af76e10b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:26.283776 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cc7eee58-050d-4956-bbc9-b703af76e10b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:26.347034 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cc7eee58-050d-4956-bbc9-b703af76e10b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:26.355843 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cc7eee58-050d-4956-bbc9-b703af76e10b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:26.357409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cc7eee58-050d-4956-bbc9-b703af76e10b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:26.358717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cc7eee58-050d-4956-bbc9-b703af76e10b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.088135 Jun 10 13:23:26.359274 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 870/1821] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:26 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:27.377380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b228bc19-0f0d-4cc2-834f-855d36c67aea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:27.378012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b228bc19-0f0d-4cc2-834f-855d36c67aea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:27.384580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b228bc19-0f0d-4cc2-834f-855d36c67aea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:27.385066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b228bc19-0f0d-4cc2-834f-855d36c67aea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:27.385819 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b228bc19-0f0d-4cc2-834f-855d36c67aea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:27.386247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b228bc19-0f0d-4cc2-834f-855d36c67aea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:27.446156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b228bc19-0f0d-4cc2-834f-855d36c67aea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:27.454368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b228bc19-0f0d-4cc2-834f-855d36c67aea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:27.458688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b228bc19-0f0d-4cc2-834f-855d36c67aea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:27.458688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b228bc19-0f0d-4cc2-834f-855d36c67aea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083799 Jun 10 13:23:27.458688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 952/1822] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:27 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:28.474785 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d00d2dad-ee48-4e07-823f-4b4cff00047a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:28.475377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d00d2dad-ee48-4e07-823f-4b4cff00047a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:28.482669 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d00d2dad-ee48-4e07-823f-4b4cff00047a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:28.483113 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d00d2dad-ee48-4e07-823f-4b4cff00047a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:28.483884 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d00d2dad-ee48-4e07-823f-4b4cff00047a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:28.484316 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d00d2dad-ee48-4e07-823f-4b4cff00047a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:28.545650 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d00d2dad-ee48-4e07-823f-4b4cff00047a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:28.554482 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d00d2dad-ee48-4e07-823f-4b4cff00047a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:28.555894 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d00d2dad-ee48-4e07-823f-4b4cff00047a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:28.557043 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d00d2dad-ee48-4e07-823f-4b4cff00047a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086972 Jun 10 13:23:28.561621 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 871/1823] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:28 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:29.575659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fd407a1f-6ff6-463d-8491-cae8f947e27f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:29.576251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fd407a1f-6ff6-463d-8491-cae8f947e27f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:29.582835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fd407a1f-6ff6-463d-8491-cae8f947e27f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:29.583284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fd407a1f-6ff6-463d-8491-cae8f947e27f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:29.584050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fd407a1f-6ff6-463d-8491-cae8f947e27f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:29.584483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fd407a1f-6ff6-463d-8491-cae8f947e27f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:29.602402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:23:29.643444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fd407a1f-6ff6-463d-8491-cae8f947e27f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:29.651506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fd407a1f-6ff6-463d-8491-cae8f947e27f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:29.653050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fd407a1f-6ff6-463d-8491-cae8f947e27f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:29.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fd407a1f-6ff6-463d-8491-cae8f947e27f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082413 Jun 10 13:23:29.654740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 953/1824] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:29 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:30.672617 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-546a81ee-d36a-4796-bff7-1ec5178a13be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:30.673173 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-546a81ee-d36a-4796-bff7-1ec5178a13be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:30.680265 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-546a81ee-d36a-4796-bff7-1ec5178a13be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:30.680717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-546a81ee-d36a-4796-bff7-1ec5178a13be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:30.681513 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-546a81ee-d36a-4796-bff7-1ec5178a13be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:30.681986 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-546a81ee-d36a-4796-bff7-1ec5178a13be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:30.743201 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-546a81ee-d36a-4796-bff7-1ec5178a13be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:30.751426 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-546a81ee-d36a-4796-bff7-1ec5178a13be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:30.753552 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-546a81ee-d36a-4796-bff7-1ec5178a13be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:30.754698 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-546a81ee-d36a-4796-bff7-1ec5178a13be tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086068 Jun 10 13:23:30.755259 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 872/1825] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:30 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:31.773087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4e3d0e60-c153-4e08-92a1-1e74a178cee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:31.773694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4e3d0e60-c153-4e08-92a1-1e74a178cee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:31.779907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4e3d0e60-c153-4e08-92a1-1e74a178cee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:31.780357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4e3d0e60-c153-4e08-92a1-1e74a178cee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:31.781141 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4e3d0e60-c153-4e08-92a1-1e74a178cee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:31.781610 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4e3d0e60-c153-4e08-92a1-1e74a178cee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:31.841239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4e3d0e60-c153-4e08-92a1-1e74a178cee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:31.850044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4e3d0e60-c153-4e08-92a1-1e74a178cee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:31.851553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4e3d0e60-c153-4e08-92a1-1e74a178cee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:31.852698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4e3d0e60-c153-4e08-92a1-1e74a178cee0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083544 Jun 10 13:23:31.853291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 954/1826] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:31 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:32.870397 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-5f85ec94-79cd-43b6-8534-5bb14b27c9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:32.871001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-5f85ec94-79cd-43b6-8534-5bb14b27c9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:32.878202 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5f85ec94-79cd-43b6-8534-5bb14b27c9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:32.878643 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5f85ec94-79cd-43b6-8534-5bb14b27c9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:32.879390 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5f85ec94-79cd-43b6-8534-5bb14b27c9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:32.879816 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-5f85ec94-79cd-43b6-8534-5bb14b27c9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:32.938082 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5f85ec94-79cd-43b6-8534-5bb14b27c9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:32.946942 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5f85ec94-79cd-43b6-8534-5bb14b27c9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:32.948408 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-5f85ec94-79cd-43b6-8534-5bb14b27c9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:32.949562 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-5f85ec94-79cd-43b6-8534-5bb14b27c9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083024 Jun 10 13:23:32.950106 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 873/1827] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:32 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:33.967483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-04ed1e2d-5869-4545-b743-a2774e74dccb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:33.968093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-04ed1e2d-5869-4545-b743-a2774e74dccb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:33.975447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04ed1e2d-5869-4545-b743-a2774e74dccb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:33.975896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04ed1e2d-5869-4545-b743-a2774e74dccb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:33.976661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04ed1e2d-5869-4545-b743-a2774e74dccb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:33.977142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04ed1e2d-5869-4545-b743-a2774e74dccb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:34.038835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-04ed1e2d-5869-4545-b743-a2774e74dccb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:34.047092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-04ed1e2d-5869-4545-b743-a2774e74dccb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:34.048588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-04ed1e2d-5869-4545-b743-a2774e74dccb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:34.049758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-04ed1e2d-5869-4545-b743-a2774e74dccb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086264 Jun 10 13:23:34.050358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 955/1828] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:33 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:34.195813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:23:34.196050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:23:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:23:35.068918 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6e270703-ffe3-43f3-bbd8-2b42771c082a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:35.069549 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6e270703-ffe3-43f3-bbd8-2b42771c082a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:35.075987 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e270703-ffe3-43f3-bbd8-2b42771c082a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:35.076427 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e270703-ffe3-43f3-bbd8-2b42771c082a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:35.077202 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e270703-ffe3-43f3-bbd8-2b42771c082a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:35.077630 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6e270703-ffe3-43f3-bbd8-2b42771c082a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:35.141108 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6e270703-ffe3-43f3-bbd8-2b42771c082a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:35.150730 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6e270703-ffe3-43f3-bbd8-2b42771c082a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:35.152216 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6e270703-ffe3-43f3-bbd8-2b42771c082a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:35.153384 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6e270703-ffe3-43f3-bbd8-2b42771c082a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.089932 Jun 10 13:23:35.153938 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 874/1829] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:35 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:36.171561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0b0c05ed-b9be-4ce9-b677-3b80856a148f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:36.172167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0b0c05ed-b9be-4ce9-b677-3b80856a148f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:36.178850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b0c05ed-b9be-4ce9-b677-3b80856a148f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:36.180177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b0c05ed-b9be-4ce9-b677-3b80856a148f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:36.180177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b0c05ed-b9be-4ce9-b677-3b80856a148f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:36.180473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0b0c05ed-b9be-4ce9-b677-3b80856a148f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:36.242188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0b0c05ed-b9be-4ce9-b677-3b80856a148f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:36.250490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0b0c05ed-b9be-4ce9-b677-3b80856a148f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:36.251993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0b0c05ed-b9be-4ce9-b677-3b80856a148f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:36.253672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0b0c05ed-b9be-4ce9-b677-3b80856a148f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086116 Jun 10 13:23:36.254228 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 956/1830] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:36 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:37.271582 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d3d47adc-39e1-4080-b7aa-30fbcccdf906 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:37.272186 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d3d47adc-39e1-4080-b7aa-30fbcccdf906 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:37.278717 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d3d47adc-39e1-4080-b7aa-30fbcccdf906 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:37.282295 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d3d47adc-39e1-4080-b7aa-30fbcccdf906 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:37.282295 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d3d47adc-39e1-4080-b7aa-30fbcccdf906 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:37.282295 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d3d47adc-39e1-4080-b7aa-30fbcccdf906 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:37.340531 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d3d47adc-39e1-4080-b7aa-30fbcccdf906 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:37.349252 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d3d47adc-39e1-4080-b7aa-30fbcccdf906 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:37.350732 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d3d47adc-39e1-4080-b7aa-30fbcccdf906 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:37.351864 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d3d47adc-39e1-4080-b7aa-30fbcccdf906 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084780 Jun 10 13:23:37.352405 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 875/1831] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:37 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:38.370418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5685b47e-fa66-4e07-95ad-ccd66ea7527b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:38.371020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5685b47e-fa66-4e07-95ad-ccd66ea7527b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:38.378253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5685b47e-fa66-4e07-95ad-ccd66ea7527b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:38.378702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5685b47e-fa66-4e07-95ad-ccd66ea7527b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:38.379491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5685b47e-fa66-4e07-95ad-ccd66ea7527b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:38.379921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5685b47e-fa66-4e07-95ad-ccd66ea7527b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:38.438519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5685b47e-fa66-4e07-95ad-ccd66ea7527b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:38.446970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5685b47e-fa66-4e07-95ad-ccd66ea7527b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:38.448456 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5685b47e-fa66-4e07-95ad-ccd66ea7527b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:38.449624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5685b47e-fa66-4e07-95ad-ccd66ea7527b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083125 Jun 10 13:23:38.450185 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 957/1832] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:38 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:39.467621 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-217ca5a1-b012-46d6-a696-7b7a61184fa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:39.468222 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-217ca5a1-b012-46d6-a696-7b7a61184fa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:39.474494 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-217ca5a1-b012-46d6-a696-7b7a61184fa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:39.474939 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-217ca5a1-b012-46d6-a696-7b7a61184fa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:39.475686 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-217ca5a1-b012-46d6-a696-7b7a61184fa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:39.476113 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-217ca5a1-b012-46d6-a696-7b7a61184fa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:39.534887 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-217ca5a1-b012-46d6-a696-7b7a61184fa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:39.543011 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-217ca5a1-b012-46d6-a696-7b7a61184fa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:39.544483 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-217ca5a1-b012-46d6-a696-7b7a61184fa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:39.545628 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-217ca5a1-b012-46d6-a696-7b7a61184fa6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082066 Jun 10 13:23:39.546178 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 876/1833] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:39 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:40.564064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a3446d5d-2bbc-474a-953d-ceb07c423cab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:40.564594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a3446d5d-2bbc-474a-953d-ceb07c423cab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:40.571791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3446d5d-2bbc-474a-953d-ceb07c423cab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:40.572242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3446d5d-2bbc-474a-953d-ceb07c423cab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:40.573038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3446d5d-2bbc-474a-953d-ceb07c423cab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:40.573475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3446d5d-2bbc-474a-953d-ceb07c423cab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:40.632955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a3446d5d-2bbc-474a-953d-ceb07c423cab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:40.641233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a3446d5d-2bbc-474a-953d-ceb07c423cab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:40.642733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a3446d5d-2bbc-474a-953d-ceb07c423cab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:40.644551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a3446d5d-2bbc-474a-953d-ceb07c423cab tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084465 Jun 10 13:23:40.645315 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 958/1834] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:40 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:41.665443 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-51ba544e-8101-4571-b132-8b51dbe2ea83 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:41.665978 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-51ba544e-8101-4571-b132-8b51dbe2ea83 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:41.672383 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-51ba544e-8101-4571-b132-8b51dbe2ea83 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:41.672864 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-51ba544e-8101-4571-b132-8b51dbe2ea83 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:41.673618 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-51ba544e-8101-4571-b132-8b51dbe2ea83 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:41.674048 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-51ba544e-8101-4571-b132-8b51dbe2ea83 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:41.735118 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-51ba544e-8101-4571-b132-8b51dbe2ea83 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:41.744052 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-51ba544e-8101-4571-b132-8b51dbe2ea83 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:41.745593 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-51ba544e-8101-4571-b132-8b51dbe2ea83 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:41.746731 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-51ba544e-8101-4571-b132-8b51dbe2ea83 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085260 Jun 10 13:23:41.747281 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 877/1835] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:41 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:42.764547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-84bfd742-5af2-487b-a2c8-688789049bb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:42.765176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-84bfd742-5af2-487b-a2c8-688789049bb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:42.772083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84bfd742-5af2-487b-a2c8-688789049bb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:42.772526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84bfd742-5af2-487b-a2c8-688789049bb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:42.773338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84bfd742-5af2-487b-a2c8-688789049bb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:42.773764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84bfd742-5af2-487b-a2c8-688789049bb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:42.835426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-84bfd742-5af2-487b-a2c8-688789049bb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:42.843686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-84bfd742-5af2-487b-a2c8-688789049bb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:42.845194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-84bfd742-5af2-487b-a2c8-688789049bb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:42.846947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-84bfd742-5af2-487b-a2c8-688789049bb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085737 Jun 10 13:23:42.847487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 959/1836] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:42 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:43.865668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d8dabd56-94fa-4a7e-a65d-f68be6139bf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:43.866267 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d8dabd56-94fa-4a7e-a65d-f68be6139bf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:43.872738 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d8dabd56-94fa-4a7e-a65d-f68be6139bf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:43.873208 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d8dabd56-94fa-4a7e-a65d-f68be6139bf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:43.873955 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d8dabd56-94fa-4a7e-a65d-f68be6139bf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:43.874383 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d8dabd56-94fa-4a7e-a65d-f68be6139bf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:43.935955 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d8dabd56-94fa-4a7e-a65d-f68be6139bf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:43.944216 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d8dabd56-94fa-4a7e-a65d-f68be6139bf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:43.945733 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d8dabd56-94fa-4a7e-a65d-f68be6139bf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:43.946850 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d8dabd56-94fa-4a7e-a65d-f68be6139bf0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085733 Jun 10 13:23:43.947402 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 878/1837] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:43 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:44.964708 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ada36589-a24b-4e1c-bd43-215130c14c21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:44.965343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ada36589-a24b-4e1c-bd43-215130c14c21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:44.971774 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ada36589-a24b-4e1c-bd43-215130c14c21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:44.972220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ada36589-a24b-4e1c-bd43-215130c14c21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:44.973274 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ada36589-a24b-4e1c-bd43-215130c14c21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:44.973706 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ada36589-a24b-4e1c-bd43-215130c14c21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:45.033213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ada36589-a24b-4e1c-bd43-215130c14c21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:45.041503 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ada36589-a24b-4e1c-bd43-215130c14c21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:45.042999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ada36589-a24b-4e1c-bd43-215130c14c21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:45.044142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ada36589-a24b-4e1c-bd43-215130c14c21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083393 Jun 10 13:23:45.044693 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 960/1838] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:44 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:46.062615 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4076be18-9bbb-4686-8c81-6a376c5a4590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:46.063222 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4076be18-9bbb-4686-8c81-6a376c5a4590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:46.069708 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4076be18-9bbb-4686-8c81-6a376c5a4590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:46.070155 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4076be18-9bbb-4686-8c81-6a376c5a4590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:46.070915 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4076be18-9bbb-4686-8c81-6a376c5a4590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:46.071347 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4076be18-9bbb-4686-8c81-6a376c5a4590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:46.133244 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4076be18-9bbb-4686-8c81-6a376c5a4590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:46.140656 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4076be18-9bbb-4686-8c81-6a376c5a4590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:46.142221 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4076be18-9bbb-4686-8c81-6a376c5a4590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:46.143309 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4076be18-9bbb-4686-8c81-6a376c5a4590 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084652 Jun 10 13:23:46.143861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 879/1839] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:46 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:47.161348 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e7aa302e-d180-4150-8e86-061deb26fb21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:47.161956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e7aa302e-d180-4150-8e86-061deb26fb21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:47.164252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=28 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:23:47.169098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7aa302e-d180-4150-8e86-061deb26fb21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:47.169548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7aa302e-d180-4150-8e86-061deb26fb21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:47.170308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7aa302e-d180-4150-8e86-061deb26fb21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:47.170746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7aa302e-d180-4150-8e86-061deb26fb21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:47.231154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e7aa302e-d180-4150-8e86-061deb26fb21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:47.239579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e7aa302e-d180-4150-8e86-061deb26fb21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:47.241064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e7aa302e-d180-4150-8e86-061deb26fb21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:47.242203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e7aa302e-d180-4150-8e86-061deb26fb21 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085410 Jun 10 13:23:47.242758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 961/1840] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:47 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:48.260435 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f6fff2d1-7430-410b-8979-b6824ffbabd6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:48.261077 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f6fff2d1-7430-410b-8979-b6824ffbabd6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:48.267592 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6fff2d1-7430-410b-8979-b6824ffbabd6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:48.268045 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6fff2d1-7430-410b-8979-b6824ffbabd6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:48.268829 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6fff2d1-7430-410b-8979-b6824ffbabd6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:48.269318 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f6fff2d1-7430-410b-8979-b6824ffbabd6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:48.273282 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=89 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:23:48.329791 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f6fff2d1-7430-410b-8979-b6824ffbabd6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:48.338260 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f6fff2d1-7430-410b-8979-b6824ffbabd6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:48.339746 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f6fff2d1-7430-410b-8979-b6824ffbabd6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:48.340924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f6fff2d1-7430-410b-8979-b6824ffbabd6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084381 Jun 10 13:23:48.341477 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 880/1841] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:48 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:49.359451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fd598296-48b3-418e-a967-f7d7afbbbe90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:49.360060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fd598296-48b3-418e-a967-f7d7afbbbe90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:49.366583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fd598296-48b3-418e-a967-f7d7afbbbe90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:49.367035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fd598296-48b3-418e-a967-f7d7afbbbe90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:49.367795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fd598296-48b3-418e-a967-f7d7afbbbe90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:49.368231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fd598296-48b3-418e-a967-f7d7afbbbe90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:49.427046 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fd598296-48b3-418e-a967-f7d7afbbbe90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:49.435369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fd598296-48b3-418e-a967-f7d7afbbbe90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:49.436902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fd598296-48b3-418e-a967-f7d7afbbbe90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:49.438079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fd598296-48b3-418e-a967-f7d7afbbbe90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082537 Jun 10 13:23:49.438637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 962/1842] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:49 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:50.455875 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-71e026f0-4e8e-4139-89ad-6662777308ec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:50.456481 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-71e026f0-4e8e-4139-89ad-6662777308ec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:50.458773 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=28 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:23:50.463519 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-71e026f0-4e8e-4139-89ad-6662777308ec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:50.463961 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-71e026f0-4e8e-4139-89ad-6662777308ec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:50.464708 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-71e026f0-4e8e-4139-89ad-6662777308ec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:50.465160 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-71e026f0-4e8e-4139-89ad-6662777308ec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:50.525131 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-71e026f0-4e8e-4139-89ad-6662777308ec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:50.533243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-71e026f0-4e8e-4139-89ad-6662777308ec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:50.534715 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-71e026f0-4e8e-4139-89ad-6662777308ec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:50.536336 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-71e026f0-4e8e-4139-89ad-6662777308ec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084363 Jun 10 13:23:50.536906 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 881/1843] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:50 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:51.554200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f662d9fe-0e6d-44f9-8fff-f5c85fdd8482 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:51.554798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f662d9fe-0e6d-44f9-8fff-f5c85fdd8482 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:51.561370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f662d9fe-0e6d-44f9-8fff-f5c85fdd8482 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:51.561807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f662d9fe-0e6d-44f9-8fff-f5c85fdd8482 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:51.562557 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f662d9fe-0e6d-44f9-8fff-f5c85fdd8482 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:51.562984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f662d9fe-0e6d-44f9-8fff-f5c85fdd8482 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:51.626223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f662d9fe-0e6d-44f9-8fff-f5c85fdd8482 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:51.635071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f662d9fe-0e6d-44f9-8fff-f5c85fdd8482 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:51.636553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f662d9fe-0e6d-44f9-8fff-f5c85fdd8482 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:51.637712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f662d9fe-0e6d-44f9-8fff-f5c85fdd8482 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087883 Jun 10 13:23:51.638264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 963/1844] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:51 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:52.656146 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-93fb54ca-5938-49e1-9656-2dac11c169d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:52.656690 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-93fb54ca-5938-49e1-9656-2dac11c169d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:52.663641 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-93fb54ca-5938-49e1-9656-2dac11c169d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:52.664083 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-93fb54ca-5938-49e1-9656-2dac11c169d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:52.664861 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-93fb54ca-5938-49e1-9656-2dac11c169d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:52.665299 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-93fb54ca-5938-49e1-9656-2dac11c169d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:52.728211 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-93fb54ca-5938-49e1-9656-2dac11c169d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:52.738936 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-93fb54ca-5938-49e1-9656-2dac11c169d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:52.740515 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-93fb54ca-5938-49e1-9656-2dac11c169d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:52.742057 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-93fb54ca-5938-49e1-9656-2dac11c169d2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.089874 Jun 10 13:23:52.742610 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 882/1845] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:52 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:53.763223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3fd8ea81-bd3a-4e48-a09b-ef8162d0ba57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:53.763842 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3fd8ea81-bd3a-4e48-a09b-ef8162d0ba57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:53.770332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3fd8ea81-bd3a-4e48-a09b-ef8162d0ba57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:53.770783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3fd8ea81-bd3a-4e48-a09b-ef8162d0ba57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:53.771544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3fd8ea81-bd3a-4e48-a09b-ef8162d0ba57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:53.771980 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3fd8ea81-bd3a-4e48-a09b-ef8162d0ba57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:53.831420 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3fd8ea81-bd3a-4e48-a09b-ef8162d0ba57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:53.840192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3fd8ea81-bd3a-4e48-a09b-ef8162d0ba57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:53.841727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3fd8ea81-bd3a-4e48-a09b-ef8162d0ba57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:53.842853 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3fd8ea81-bd3a-4e48-a09b-ef8162d0ba57 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083588 Jun 10 13:23:53.843404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 964/1846] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:53 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:54.861454 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f781be18-2e21-49ca-bb90-e1286a09b47b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:54.862059 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f781be18-2e21-49ca-bb90-e1286a09b47b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:54.869125 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f781be18-2e21-49ca-bb90-e1286a09b47b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:54.869580 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f781be18-2e21-49ca-bb90-e1286a09b47b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:54.870338 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f781be18-2e21-49ca-bb90-e1286a09b47b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:54.870770 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f781be18-2e21-49ca-bb90-e1286a09b47b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:54.932079 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f781be18-2e21-49ca-bb90-e1286a09b47b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:54.940638 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f781be18-2e21-49ca-bb90-e1286a09b47b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:54.942672 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f781be18-2e21-49ca-bb90-e1286a09b47b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:54.943876 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f781be18-2e21-49ca-bb90-e1286a09b47b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086294 Jun 10 13:23:54.944391 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 883/1847] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:54 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:55.962602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fcc62b1b-9b34-4b3d-bc47-38afd83bdff3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:55.963206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fcc62b1b-9b34-4b3d-bc47-38afd83bdff3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:55.969688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fcc62b1b-9b34-4b3d-bc47-38afd83bdff3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:55.970132 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fcc62b1b-9b34-4b3d-bc47-38afd83bdff3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:55.970899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fcc62b1b-9b34-4b3d-bc47-38afd83bdff3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:55.971312 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fcc62b1b-9b34-4b3d-bc47-38afd83bdff3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:56.031872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fcc62b1b-9b34-4b3d-bc47-38afd83bdff3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:56.040118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fcc62b1b-9b34-4b3d-bc47-38afd83bdff3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:56.041648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fcc62b1b-9b34-4b3d-bc47-38afd83bdff3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:56.042786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fcc62b1b-9b34-4b3d-bc47-38afd83bdff3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084255 Jun 10 13:23:56.043336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 965/1848] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:55 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:57.060296 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ccc3b723-39e2-4cf5-925f-b6a226ca3418 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:57.060967 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ccc3b723-39e2-4cf5-925f-b6a226ca3418 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:57.067481 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ccc3b723-39e2-4cf5-925f-b6a226ca3418 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:57.067930 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ccc3b723-39e2-4cf5-925f-b6a226ca3418 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:57.068696 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ccc3b723-39e2-4cf5-925f-b6a226ca3418 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:57.069155 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ccc3b723-39e2-4cf5-925f-b6a226ca3418 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:57.129412 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ccc3b723-39e2-4cf5-925f-b6a226ca3418 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:57.137736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ccc3b723-39e2-4cf5-925f-b6a226ca3418 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:57.139234 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ccc3b723-39e2-4cf5-925f-b6a226ca3418 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:57.140370 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ccc3b723-39e2-4cf5-925f-b6a226ca3418 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083991 Jun 10 13:23:57.140957 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 884/1849] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:57 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:58.158888 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e6ea06bf-5704-4a45-964f-4815f9a9d79f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:58.160011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e6ea06bf-5704-4a45-964f-4815f9a9d79f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:58.166490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e6ea06bf-5704-4a45-964f-4815f9a9d79f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:58.169088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e6ea06bf-5704-4a45-964f-4815f9a9d79f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:58.169088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e6ea06bf-5704-4a45-964f-4815f9a9d79f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:58.169088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e6ea06bf-5704-4a45-964f-4815f9a9d79f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:58.227449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e6ea06bf-5704-4a45-964f-4815f9a9d79f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:58.235819 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e6ea06bf-5704-4a45-964f-4815f9a9d79f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:58.237329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e6ea06bf-5704-4a45-964f-4815f9a9d79f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:58.239044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e6ea06bf-5704-4a45-964f-4815f9a9d79f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084107 Jun 10 13:23:58.239594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 966/1850] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:58 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:23:59.259417 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-46f032f1-ada5-4cfc-9f9e-7a5204e91525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:23:59.260027 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-46f032f1-ada5-4cfc-9f9e-7a5204e91525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:23:59.266640 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46f032f1-ada5-4cfc-9f9e-7a5204e91525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:59.267086 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46f032f1-ada5-4cfc-9f9e-7a5204e91525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:59.267847 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46f032f1-ada5-4cfc-9f9e-7a5204e91525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:23:59.268276 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-46f032f1-ada5-4cfc-9f9e-7a5204e91525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:23:59.330561 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-46f032f1-ada5-4cfc-9f9e-7a5204e91525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:59.339287 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-46f032f1-ada5-4cfc-9f9e-7a5204e91525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:59.340792 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-46f032f1-ada5-4cfc-9f9e-7a5204e91525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:23:59.341926 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-46f032f1-ada5-4cfc-9f9e-7a5204e91525 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086472 Jun 10 13:23:59.342491 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 885/1851] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:23:59 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:00.360004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-239e1d16-fb37-425e-a876-797478b3540c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:00.360608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-239e1d16-fb37-425e-a876-797478b3540c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:00.367060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-239e1d16-fb37-425e-a876-797478b3540c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:00.367519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-239e1d16-fb37-425e-a876-797478b3540c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:00.368250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-239e1d16-fb37-425e-a876-797478b3540c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:00.368676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-239e1d16-fb37-425e-a876-797478b3540c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:00.372502 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=81 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:24:00.435042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-239e1d16-fb37-425e-a876-797478b3540c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:00.443972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-239e1d16-fb37-425e-a876-797478b3540c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:00.445493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-239e1d16-fb37-425e-a876-797478b3540c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:00.446620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-239e1d16-fb37-425e-a876-797478b3540c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.090511 Jun 10 13:24:00.447170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 967/1852] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:00 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:01.465374 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-cd0afc68-302b-4680-b609-7958d44ef5e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:01.466096 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-cd0afc68-302b-4680-b609-7958d44ef5e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:01.473559 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cd0afc68-302b-4680-b609-7958d44ef5e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:01.474246 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cd0afc68-302b-4680-b609-7958d44ef5e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:01.475014 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cd0afc68-302b-4680-b609-7958d44ef5e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:01.475442 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-cd0afc68-302b-4680-b609-7958d44ef5e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:01.544308 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cd0afc68-302b-4680-b609-7958d44ef5e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:01.553167 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cd0afc68-302b-4680-b609-7958d44ef5e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:01.554653 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-cd0afc68-302b-4680-b609-7958d44ef5e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:01.555786 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-cd0afc68-302b-4680-b609-7958d44ef5e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.094378 Jun 10 13:24:01.556338 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 886/1853] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:01 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:02.574254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5d030438-d7bc-4de6-93b0-a8f52bff20da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:02.574763 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5d030438-d7bc-4de6-93b0-a8f52bff20da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:02.581106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d030438-d7bc-4de6-93b0-a8f52bff20da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:02.581561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d030438-d7bc-4de6-93b0-a8f52bff20da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:02.582322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d030438-d7bc-4de6-93b0-a8f52bff20da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:02.582757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d030438-d7bc-4de6-93b0-a8f52bff20da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:02.641991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5d030438-d7bc-4de6-93b0-a8f52bff20da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:02.652118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5d030438-d7bc-4de6-93b0-a8f52bff20da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:02.653654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5d030438-d7bc-4de6-93b0-a8f52bff20da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:02.654808 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5d030438-d7bc-4de6-93b0-a8f52bff20da tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084568 Jun 10 13:24:02.655365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 968/1854] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:02 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:03.672946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2367876d-92d7-4cfa-ab8c-377b195213e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:03.673496 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2367876d-92d7-4cfa-ab8c-377b195213e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:03.680008 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2367876d-92d7-4cfa-ab8c-377b195213e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:03.680452 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2367876d-92d7-4cfa-ab8c-377b195213e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:03.681240 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2367876d-92d7-4cfa-ab8c-377b195213e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:03.681676 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2367876d-92d7-4cfa-ab8c-377b195213e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:03.741410 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2367876d-92d7-4cfa-ab8c-377b195213e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:03.749748 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2367876d-92d7-4cfa-ab8c-377b195213e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:03.751218 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2367876d-92d7-4cfa-ab8c-377b195213e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:03.752344 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2367876d-92d7-4cfa-ab8c-377b195213e6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083489 Jun 10 13:24:03.752928 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 887/1855] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:03 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:04.769235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-26a8f42d-961d-4413-aa47-44ce153d41f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:04.769848 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-26a8f42d-961d-4413-aa47-44ce153d41f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:04.776223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26a8f42d-961d-4413-aa47-44ce153d41f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:04.776666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26a8f42d-961d-4413-aa47-44ce153d41f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:04.777459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26a8f42d-961d-4413-aa47-44ce153d41f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:04.777886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26a8f42d-961d-4413-aa47-44ce153d41f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:04.836604 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-26a8f42d-961d-4413-aa47-44ce153d41f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:04.845217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-26a8f42d-961d-4413-aa47-44ce153d41f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:04.847680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-26a8f42d-961d-4413-aa47-44ce153d41f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:04.848932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-26a8f42d-961d-4413-aa47-44ce153d41f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083564 Jun 10 13:24:04.849491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 969/1856] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:04 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:05.867371 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3fd1880d-a5f7-417e-b46f-4bfd285e985d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:05.867974 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3fd1880d-a5f7-417e-b46f-4bfd285e985d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:05.874558 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3fd1880d-a5f7-417e-b46f-4bfd285e985d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:05.875008 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3fd1880d-a5f7-417e-b46f-4bfd285e985d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:05.875781 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3fd1880d-a5f7-417e-b46f-4bfd285e985d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:05.876216 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3fd1880d-a5f7-417e-b46f-4bfd285e985d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:05.935993 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3fd1880d-a5f7-417e-b46f-4bfd285e985d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:05.945033 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3fd1880d-a5f7-417e-b46f-4bfd285e985d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:05.946520 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3fd1880d-a5f7-417e-b46f-4bfd285e985d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:05.947654 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3fd1880d-a5f7-417e-b46f-4bfd285e985d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084225 Jun 10 13:24:05.948204 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 888/1857] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:05 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:06.966559 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b49f2655-3769-4b5a-b3e4-41c997db655f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:06.967170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b49f2655-3769-4b5a-b3e4-41c997db655f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:06.973860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b49f2655-3769-4b5a-b3e4-41c997db655f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:06.974311 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b49f2655-3769-4b5a-b3e4-41c997db655f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:06.975067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b49f2655-3769-4b5a-b3e4-41c997db655f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:06.975506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b49f2655-3769-4b5a-b3e4-41c997db655f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:07.036410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b49f2655-3769-4b5a-b3e4-41c997db655f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:07.044732 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b49f2655-3769-4b5a-b3e4-41c997db655f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:07.046248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b49f2655-3769-4b5a-b3e4-41c997db655f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:07.047385 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b49f2655-3769-4b5a-b3e4-41c997db655f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084739 Jun 10 13:24:07.047942 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 970/1858] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:06 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:08.066530 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-2d77c1f1-6ecb-4b7c-b10e-0a9f4b4e78d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:08.067136 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-2d77c1f1-6ecb-4b7c-b10e-0a9f4b4e78d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:08.073195 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d77c1f1-6ecb-4b7c-b10e-0a9f4b4e78d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:08.073643 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d77c1f1-6ecb-4b7c-b10e-0a9f4b4e78d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:08.074400 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d77c1f1-6ecb-4b7c-b10e-0a9f4b4e78d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:08.074833 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-2d77c1f1-6ecb-4b7c-b10e-0a9f4b4e78d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:08.136174 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2d77c1f1-6ecb-4b7c-b10e-0a9f4b4e78d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:08.145009 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2d77c1f1-6ecb-4b7c-b10e-0a9f4b4e78d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:08.146505 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-2d77c1f1-6ecb-4b7c-b10e-0a9f4b4e78d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:08.147637 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-2d77c1f1-6ecb-4b7c-b10e-0a9f4b4e78d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085525 Jun 10 13:24:08.148188 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 889/1859] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:08 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:09.166091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-01ea85c9-01f2-40be-bd12-0b3fa8e7b9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:09.166693 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-01ea85c9-01f2-40be-bd12-0b3fa8e7b9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:09.173361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01ea85c9-01f2-40be-bd12-0b3fa8e7b9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:09.173804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01ea85c9-01f2-40be-bd12-0b3fa8e7b9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:09.174556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01ea85c9-01f2-40be-bd12-0b3fa8e7b9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:09.174985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01ea85c9-01f2-40be-bd12-0b3fa8e7b9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:09.238581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-01ea85c9-01f2-40be-bd12-0b3fa8e7b9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:09.247097 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-01ea85c9-01f2-40be-bd12-0b3fa8e7b9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:09.248586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-01ea85c9-01f2-40be-bd12-0b3fa8e7b9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:09.249741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-01ea85c9-01f2-40be-bd12-0b3fa8e7b9c5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087685 Jun 10 13:24:09.250393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 971/1860] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:09 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:10.267670 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f14513cd-8f75-49ed-a448-9312a77bc034 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:10.269001 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f14513cd-8f75-49ed-a448-9312a77bc034 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:10.275202 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f14513cd-8f75-49ed-a448-9312a77bc034 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:10.275725 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f14513cd-8f75-49ed-a448-9312a77bc034 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:10.276590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f14513cd-8f75-49ed-a448-9312a77bc034 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:10.277049 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f14513cd-8f75-49ed-a448-9312a77bc034 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:10.341048 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f14513cd-8f75-49ed-a448-9312a77bc034 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:10.350153 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f14513cd-8f75-49ed-a448-9312a77bc034 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:10.352319 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f14513cd-8f75-49ed-a448-9312a77bc034 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:10.353538 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f14513cd-8f75-49ed-a448-9312a77bc034 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.090002 Jun 10 13:24:10.357831 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 890/1861] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:10 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:11.371518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fbbed861-c8d0-43b0-875e-68bfa6e4f4d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:11.372130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fbbed861-c8d0-43b0-875e-68bfa6e4f4d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:11.378772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fbbed861-c8d0-43b0-875e-68bfa6e4f4d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:11.379212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fbbed861-c8d0-43b0-875e-68bfa6e4f4d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:11.379967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fbbed861-c8d0-43b0-875e-68bfa6e4f4d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:11.380397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fbbed861-c8d0-43b0-875e-68bfa6e4f4d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:11.442552 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fbbed861-c8d0-43b0-875e-68bfa6e4f4d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:11.451116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fbbed861-c8d0-43b0-875e-68bfa6e4f4d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:11.452660 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fbbed861-c8d0-43b0-875e-68bfa6e4f4d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:11.453824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fbbed861-c8d0-43b0-875e-68bfa6e4f4d9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086244 Jun 10 13:24:11.454396 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 972/1862] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:11 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:12.471692 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c20ea011-040d-4362-8bc2-62ede4fba166 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:12.472548 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c20ea011-040d-4362-8bc2-62ede4fba166 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:12.479090 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c20ea011-040d-4362-8bc2-62ede4fba166 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:12.479532 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c20ea011-040d-4362-8bc2-62ede4fba166 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:12.480278 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c20ea011-040d-4362-8bc2-62ede4fba166 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:12.480700 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c20ea011-040d-4362-8bc2-62ede4fba166 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:12.539465 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c20ea011-040d-4362-8bc2-62ede4fba166 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:12.547627 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c20ea011-040d-4362-8bc2-62ede4fba166 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:12.549122 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c20ea011-040d-4362-8bc2-62ede4fba166 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:12.550780 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c20ea011-040d-4362-8bc2-62ede4fba166 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083013 Jun 10 13:24:12.551332 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 891/1863] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:12 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:13.568632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-15ea9814-a900-46ba-a85d-8d155c254e63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:13.569268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-15ea9814-a900-46ba-a85d-8d155c254e63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:13.575988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15ea9814-a900-46ba-a85d-8d155c254e63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:13.576435 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15ea9814-a900-46ba-a85d-8d155c254e63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:13.577227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15ea9814-a900-46ba-a85d-8d155c254e63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:13.577664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15ea9814-a900-46ba-a85d-8d155c254e63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:13.636891 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-15ea9814-a900-46ba-a85d-8d155c254e63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:13.645766 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-15ea9814-a900-46ba-a85d-8d155c254e63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:13.647263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-15ea9814-a900-46ba-a85d-8d155c254e63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:13.648399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-15ea9814-a900-46ba-a85d-8d155c254e63 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083741 Jun 10 13:24:13.648985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 973/1864] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:13 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:14.666226 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-28eedd67-0ffc-4e30-b1c4-241ff84e1eef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:14.666760 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-28eedd67-0ffc-4e30-b1c4-241ff84e1eef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:14.673261 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-28eedd67-0ffc-4e30-b1c4-241ff84e1eef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:14.673707 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-28eedd67-0ffc-4e30-b1c4-241ff84e1eef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:14.674467 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-28eedd67-0ffc-4e30-b1c4-241ff84e1eef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:14.674953 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-28eedd67-0ffc-4e30-b1c4-241ff84e1eef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:14.736520 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-28eedd67-0ffc-4e30-b1c4-241ff84e1eef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:14.745181 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-28eedd67-0ffc-4e30-b1c4-241ff84e1eef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:14.746683 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-28eedd67-0ffc-4e30-b1c4-241ff84e1eef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:14.749792 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-28eedd67-0ffc-4e30-b1c4-241ff84e1eef tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087623 Jun 10 13:24:14.751446 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 892/1865] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:14 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:15.769620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-48e47465-3354-4c55-a9d2-00875b4339a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:15.770220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-48e47465-3354-4c55-a9d2-00875b4339a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:15.776609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48e47465-3354-4c55-a9d2-00875b4339a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:15.777083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48e47465-3354-4c55-a9d2-00875b4339a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:15.777834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48e47465-3354-4c55-a9d2-00875b4339a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:15.778262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48e47465-3354-4c55-a9d2-00875b4339a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:15.838243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-48e47465-3354-4c55-a9d2-00875b4339a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:15.846827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-48e47465-3354-4c55-a9d2-00875b4339a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:15.848583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-48e47465-3354-4c55-a9d2-00875b4339a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:15.849754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-48e47465-3354-4c55-a9d2-00875b4339a4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084123 Jun 10 13:24:15.850352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 974/1866] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:15 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:16.868051 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f7a3be66-155f-4a5f-8d91-611097f12068 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:16.868657 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f7a3be66-155f-4a5f-8d91-611097f12068 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:16.875304 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f7a3be66-155f-4a5f-8d91-611097f12068 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:16.875747 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f7a3be66-155f-4a5f-8d91-611097f12068 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:16.876504 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f7a3be66-155f-4a5f-8d91-611097f12068 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:16.876977 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f7a3be66-155f-4a5f-8d91-611097f12068 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:16.937316 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f7a3be66-155f-4a5f-8d91-611097f12068 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:16.945674 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f7a3be66-155f-4a5f-8d91-611097f12068 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:16.947160 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f7a3be66-155f-4a5f-8d91-611097f12068 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:16.948300 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f7a3be66-155f-4a5f-8d91-611097f12068 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084172 Jun 10 13:24:16.948881 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 893/1867] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:16 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:17.966405 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ee04519e-e768-42a9-bb50-821a35d27918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:17.966997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ee04519e-e768-42a9-bb50-821a35d27918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:17.969364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=28 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:24:17.975017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee04519e-e768-42a9-bb50-821a35d27918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:17.975463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee04519e-e768-42a9-bb50-821a35d27918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:17.976217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee04519e-e768-42a9-bb50-821a35d27918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:17.976648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee04519e-e768-42a9-bb50-821a35d27918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:18.038703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ee04519e-e768-42a9-bb50-821a35d27918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:18.047010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ee04519e-e768-42a9-bb50-821a35d27918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:18.048520 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ee04519e-e768-42a9-bb50-821a35d27918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:18.049697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ee04519e-e768-42a9-bb50-821a35d27918 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087246 Jun 10 13:24:18.050259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 975/1868] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:17 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:19.067458 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-ce4c53d9-c7b0-498e-8b70-a9dcea0e402f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:19.068064 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-ce4c53d9-c7b0-498e-8b70-a9dcea0e402f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:19.074552 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ce4c53d9-c7b0-498e-8b70-a9dcea0e402f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:19.075000 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ce4c53d9-c7b0-498e-8b70-a9dcea0e402f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:19.075771 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ce4c53d9-c7b0-498e-8b70-a9dcea0e402f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:19.076243 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-ce4c53d9-c7b0-498e-8b70-a9dcea0e402f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:19.080090 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=84 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:24:19.140450 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ce4c53d9-c7b0-498e-8b70-a9dcea0e402f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:19.148929 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ce4c53d9-c7b0-498e-8b70-a9dcea0e402f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:19.150423 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-ce4c53d9-c7b0-498e-8b70-a9dcea0e402f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:19.151562 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-ce4c53d9-c7b0-498e-8b70-a9dcea0e402f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087989 Jun 10 13:24:19.152108 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 894/1869] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:19 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:20.170093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-200e7e8a-1643-4305-a0b0-12590f9a487d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:20.170698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-200e7e8a-1643-4305-a0b0-12590f9a487d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:20.177735 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-200e7e8a-1643-4305-a0b0-12590f9a487d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:20.178189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-200e7e8a-1643-4305-a0b0-12590f9a487d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:20.178945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-200e7e8a-1643-4305-a0b0-12590f9a487d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:20.179407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-200e7e8a-1643-4305-a0b0-12590f9a487d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:20.242230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-200e7e8a-1643-4305-a0b0-12590f9a487d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:20.250741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-200e7e8a-1643-4305-a0b0-12590f9a487d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:20.252917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-200e7e8a-1643-4305-a0b0-12590f9a487d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:20.254086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-200e7e8a-1643-4305-a0b0-12590f9a487d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087936 Jun 10 13:24:20.254661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 976/1870] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:20 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:21.272409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-1b8fdb60-1c62-4695-8fbc-77f83497a483 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:21.273044 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-1b8fdb60-1c62-4695-8fbc-77f83497a483 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:21.275394 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=28 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:24:21.280112 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1b8fdb60-1c62-4695-8fbc-77f83497a483 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:21.280557 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1b8fdb60-1c62-4695-8fbc-77f83497a483 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:21.281358 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1b8fdb60-1c62-4695-8fbc-77f83497a483 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:21.281796 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-1b8fdb60-1c62-4695-8fbc-77f83497a483 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:21.344420 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1b8fdb60-1c62-4695-8fbc-77f83497a483 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:21.353640 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1b8fdb60-1c62-4695-8fbc-77f83497a483 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:21.355124 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-1b8fdb60-1c62-4695-8fbc-77f83497a483 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:21.356264 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-1b8fdb60-1c62-4695-8fbc-77f83497a483 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.088329 Jun 10 13:24:21.356834 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 895/1871] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:21 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:22.374301 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-779fa749-b0bd-40da-950d-8d02c4198755 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:22.374997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-779fa749-b0bd-40da-950d-8d02c4198755 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:22.381501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-779fa749-b0bd-40da-950d-8d02c4198755 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:22.382042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-779fa749-b0bd-40da-950d-8d02c4198755 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:22.382865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-779fa749-b0bd-40da-950d-8d02c4198755 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:22.383298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-779fa749-b0bd-40da-950d-8d02c4198755 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:22.445617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-779fa749-b0bd-40da-950d-8d02c4198755 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:22.453954 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-779fa749-b0bd-40da-950d-8d02c4198755 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:22.455938 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-779fa749-b0bd-40da-950d-8d02c4198755 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:22.457096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-779fa749-b0bd-40da-950d-8d02c4198755 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086715 Jun 10 13:24:22.457649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 977/1872] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:22 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:23.475799 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-957de218-3a4e-44db-8a90-aa7d5784b516 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:23.476401 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-957de218-3a4e-44db-8a90-aa7d5784b516 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:23.482878 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-957de218-3a4e-44db-8a90-aa7d5784b516 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:23.483317 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-957de218-3a4e-44db-8a90-aa7d5784b516 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:23.484063 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-957de218-3a4e-44db-8a90-aa7d5784b516 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:23.484490 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-957de218-3a4e-44db-8a90-aa7d5784b516 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:23.544011 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-957de218-3a4e-44db-8a90-aa7d5784b516 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:23.552795 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-957de218-3a4e-44db-8a90-aa7d5784b516 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:23.554284 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-957de218-3a4e-44db-8a90-aa7d5784b516 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:23.555419 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-957de218-3a4e-44db-8a90-aa7d5784b516 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083749 Jun 10 13:24:23.555977 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 896/1873] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:23 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:24.573249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-93b369a9-d29f-4289-ba53-fb824a584d75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:24.573859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-93b369a9-d29f-4289-ba53-fb824a584d75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:24.581031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-93b369a9-d29f-4289-ba53-fb824a584d75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:24.581420 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-93b369a9-d29f-4289-ba53-fb824a584d75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:24.582174 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-93b369a9-d29f-4289-ba53-fb824a584d75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:24.582609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-93b369a9-d29f-4289-ba53-fb824a584d75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:24.641349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-93b369a9-d29f-4289-ba53-fb824a584d75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:24.650096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-93b369a9-d29f-4289-ba53-fb824a584d75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:24.651602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-93b369a9-d29f-4289-ba53-fb824a584d75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:24.652738 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-93b369a9-d29f-4289-ba53-fb824a584d75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083365 Jun 10 13:24:24.653309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 978/1874] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:24 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:25.671428 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7612c7ae-b74b-4c3d-87be-89872757a779 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:25.671978 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7612c7ae-b74b-4c3d-87be-89872757a779 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:25.678375 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7612c7ae-b74b-4c3d-87be-89872757a779 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:25.678813 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7612c7ae-b74b-4c3d-87be-89872757a779 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:25.679564 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7612c7ae-b74b-4c3d-87be-89872757a779 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:25.679986 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7612c7ae-b74b-4c3d-87be-89872757a779 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:25.742924 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7612c7ae-b74b-4c3d-87be-89872757a779 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:25.747582 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7612c7ae-b74b-4c3d-87be-89872757a779 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:25.749077 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7612c7ae-b74b-4c3d-87be-89872757a779 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:25.750323 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7612c7ae-b74b-4c3d-87be-89872757a779 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082881 Jun 10 13:24:25.750873 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 897/1875] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:25 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:26.768048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bb5906d6-c9f5-4fb9-8c97-392d41fa4c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:26.768664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bb5906d6-c9f5-4fb9-8c97-392d41fa4c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:26.775175 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb5906d6-c9f5-4fb9-8c97-392d41fa4c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:26.775623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb5906d6-c9f5-4fb9-8c97-392d41fa4c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:26.776379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb5906d6-c9f5-4fb9-8c97-392d41fa4c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:26.776835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb5906d6-c9f5-4fb9-8c97-392d41fa4c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:26.839036 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bb5906d6-c9f5-4fb9-8c97-392d41fa4c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:26.847652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bb5906d6-c9f5-4fb9-8c97-392d41fa4c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:26.849156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bb5906d6-c9f5-4fb9-8c97-392d41fa4c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:26.850294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bb5906d6-c9f5-4fb9-8c97-392d41fa4c08 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086772 Jun 10 13:24:26.850841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 979/1876] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:26 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:27.868189 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-01975ae6-84e0-4577-b173-5e33e9c60727 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:27.868832 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-01975ae6-84e0-4577-b173-5e33e9c60727 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:27.875338 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-01975ae6-84e0-4577-b173-5e33e9c60727 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:27.875783 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-01975ae6-84e0-4577-b173-5e33e9c60727 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:27.876544 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-01975ae6-84e0-4577-b173-5e33e9c60727 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:27.877021 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-01975ae6-84e0-4577-b173-5e33e9c60727 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:27.939825 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-01975ae6-84e0-4577-b173-5e33e9c60727 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:27.948005 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-01975ae6-84e0-4577-b173-5e33e9c60727 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:27.949572 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-01975ae6-84e0-4577-b173-5e33e9c60727 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:27.950744 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-01975ae6-84e0-4577-b173-5e33e9c60727 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086464 Jun 10 13:24:27.951293 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 898/1877] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:27 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:28.968990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3f06a122-cce7-48e7-896a-2a2ace0d33f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:28.969599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3f06a122-cce7-48e7-896a-2a2ace0d33f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:28.976277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f06a122-cce7-48e7-896a-2a2ace0d33f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:28.976730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f06a122-cce7-48e7-896a-2a2ace0d33f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:28.977516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f06a122-cce7-48e7-896a-2a2ace0d33f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:28.977953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f06a122-cce7-48e7-896a-2a2ace0d33f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:29.038648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f06a122-cce7-48e7-896a-2a2ace0d33f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:29.049340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f06a122-cce7-48e7-896a-2a2ace0d33f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:29.050967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f06a122-cce7-48e7-896a-2a2ace0d33f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:29.052186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3f06a122-cce7-48e7-896a-2a2ace0d33f5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087079 Jun 10 13:24:29.052751 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 980/1878] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:28 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:30.071381 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a442ae00-930e-41a9-bb85-3672e7ba48b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:30.072000 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a442ae00-930e-41a9-bb85-3672e7ba48b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:30.081376 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a442ae00-930e-41a9-bb85-3672e7ba48b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:30.081828 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a442ae00-930e-41a9-bb85-3672e7ba48b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:30.082611 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a442ae00-930e-41a9-bb85-3672e7ba48b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:30.083040 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a442ae00-930e-41a9-bb85-3672e7ba48b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:30.144463 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a442ae00-930e-41a9-bb85-3672e7ba48b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:30.152678 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a442ae00-930e-41a9-bb85-3672e7ba48b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:30.154227 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a442ae00-930e-41a9-bb85-3672e7ba48b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:30.155362 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-a442ae00-930e-41a9-bb85-3672e7ba48b5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.088711 Jun 10 13:24:30.155913 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 899/1879] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:30 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:31.173649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6844430b-fbf8-4b75-acb0-43c231461c18 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:31.174295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6844430b-fbf8-4b75-acb0-43c231461c18 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:31.180434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6844430b-fbf8-4b75-acb0-43c231461c18 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:31.180921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6844430b-fbf8-4b75-acb0-43c231461c18 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:31.181696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6844430b-fbf8-4b75-acb0-43c231461c18 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:31.182134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6844430b-fbf8-4b75-acb0-43c231461c18 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:31.184650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=84 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:24:31.241648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6844430b-fbf8-4b75-acb0-43c231461c18 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:31.249754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6844430b-fbf8-4b75-acb0-43c231461c18 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:31.251246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6844430b-fbf8-4b75-acb0-43c231461c18 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:31.252971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6844430b-fbf8-4b75-acb0-43c231461c18 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083158 Jun 10 13:24:31.253527 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 981/1880] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:31 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:32.270797 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-f7bae13e-f6de-46f1-bd5b-94ea1d4222d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:32.271405 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-f7bae13e-f6de-46f1-bd5b-94ea1d4222d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:32.278426 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f7bae13e-f6de-46f1-bd5b-94ea1d4222d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:32.278868 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f7bae13e-f6de-46f1-bd5b-94ea1d4222d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:32.279624 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f7bae13e-f6de-46f1-bd5b-94ea1d4222d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:32.280057 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-f7bae13e-f6de-46f1-bd5b-94ea1d4222d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:32.344365 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f7bae13e-f6de-46f1-bd5b-94ea1d4222d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:32.350778 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f7bae13e-f6de-46f1-bd5b-94ea1d4222d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:32.352267 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-f7bae13e-f6de-46f1-bd5b-94ea1d4222d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:32.353485 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-f7bae13e-f6de-46f1-bd5b-94ea1d4222d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086665 Jun 10 13:24:32.354212 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 900/1881] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:32 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:33.371830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b3a615e3-998e-4b7d-af3e-4a6e00208a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:33.372439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b3a615e3-998e-4b7d-af3e-4a6e00208a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:33.379102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3a615e3-998e-4b7d-af3e-4a6e00208a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:33.379783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3a615e3-998e-4b7d-af3e-4a6e00208a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:33.380354 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3a615e3-998e-4b7d-af3e-4a6e00208a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:33.380841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3a615e3-998e-4b7d-af3e-4a6e00208a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:33.444431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b3a615e3-998e-4b7d-af3e-4a6e00208a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:33.452901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b3a615e3-998e-4b7d-af3e-4a6e00208a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:33.457794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b3a615e3-998e-4b7d-af3e-4a6e00208a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:33.457794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b3a615e3-998e-4b7d-af3e-4a6e00208a07 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.088073 Jun 10 13:24:33.457794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 982/1882] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:33 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:34.474037 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-9afcc305-6120-494d-bd2f-6801e7251e79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:34.474641 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-9afcc305-6120-494d-bd2f-6801e7251e79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:34.481206 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9afcc305-6120-494d-bd2f-6801e7251e79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:34.481650 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9afcc305-6120-494d-bd2f-6801e7251e79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:34.482400 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9afcc305-6120-494d-bd2f-6801e7251e79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:34.482832 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-9afcc305-6120-494d-bd2f-6801e7251e79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:34.541553 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9afcc305-6120-494d-bd2f-6801e7251e79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:34.550158 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9afcc305-6120-494d-bd2f-6801e7251e79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:34.551631 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-9afcc305-6120-494d-bd2f-6801e7251e79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:34.552807 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-9afcc305-6120-494d-bd2f-6801e7251e79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082954 Jun 10 13:24:34.553362 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 901/1883] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:34 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:35.570570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5edd66f8-f2c9-4c4b-98d9-c492a5736410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:35.571187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5edd66f8-f2c9-4c4b-98d9-c492a5736410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:35.577777 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5edd66f8-f2c9-4c4b-98d9-c492a5736410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:35.578231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5edd66f8-f2c9-4c4b-98d9-c492a5736410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:35.578995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5edd66f8-f2c9-4c4b-98d9-c492a5736410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:35.579432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5edd66f8-f2c9-4c4b-98d9-c492a5736410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:35.643384 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5edd66f8-f2c9-4c4b-98d9-c492a5736410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:35.651668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5edd66f8-f2c9-4c4b-98d9-c492a5736410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:35.653207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5edd66f8-f2c9-4c4b-98d9-c492a5736410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:35.654351 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5edd66f8-f2c9-4c4b-98d9-c492a5736410 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087792 Jun 10 13:24:35.654902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 983/1884] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:35 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:36.672808 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-159a0c84-1177-40fe-9f79-5c132fede72f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:36.673342 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-159a0c84-1177-40fe-9f79-5c132fede72f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:36.679729 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-159a0c84-1177-40fe-9f79-5c132fede72f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:36.680174 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-159a0c84-1177-40fe-9f79-5c132fede72f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:36.680964 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-159a0c84-1177-40fe-9f79-5c132fede72f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:36.681399 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-159a0c84-1177-40fe-9f79-5c132fede72f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:36.743473 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-159a0c84-1177-40fe-9f79-5c132fede72f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:36.751740 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-159a0c84-1177-40fe-9f79-5c132fede72f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:36.753268 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-159a0c84-1177-40fe-9f79-5c132fede72f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:36.754402 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-159a0c84-1177-40fe-9f79-5c132fede72f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085591 Jun 10 13:24:36.754963 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 902/1885] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:36 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:37.772757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e4c553fe-93a5-44fe-b139-c226f4ab55dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:37.773356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e4c553fe-93a5-44fe-b139-c226f4ab55dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:37.779924 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4c553fe-93a5-44fe-b139-c226f4ab55dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:37.780371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4c553fe-93a5-44fe-b139-c226f4ab55dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:37.781176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4c553fe-93a5-44fe-b139-c226f4ab55dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:37.781613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4c553fe-93a5-44fe-b139-c226f4ab55dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:37.840655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e4c553fe-93a5-44fe-b139-c226f4ab55dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:37.849299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e4c553fe-93a5-44fe-b139-c226f4ab55dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:37.850796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e4c553fe-93a5-44fe-b139-c226f4ab55dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:37.851929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e4c553fe-93a5-44fe-b139-c226f4ab55dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083037 Jun 10 13:24:37.852474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 984/1886] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:37 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:38.869738 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-445e626d-643c-4acf-be18-16d29a31d3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:38.870354 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-445e626d-643c-4acf-be18-16d29a31d3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:38.876933 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-445e626d-643c-4acf-be18-16d29a31d3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:38.877329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-445e626d-643c-4acf-be18-16d29a31d3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:38.878078 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-445e626d-643c-4acf-be18-16d29a31d3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:38.878505 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-445e626d-643c-4acf-be18-16d29a31d3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:38.937675 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-445e626d-643c-4acf-be18-16d29a31d3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:38.945946 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-445e626d-643c-4acf-be18-16d29a31d3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:38.947415 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-445e626d-643c-4acf-be18-16d29a31d3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:38.948536 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-445e626d-643c-4acf-be18-16d29a31d3b6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082755 Jun 10 13:24:38.949116 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 903/1887] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:38 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:39.966930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1d383cab-e43a-46d7-a3d8-c4b281959210 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:39.967534 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1d383cab-e43a-46d7-a3d8-c4b281959210 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:39.974903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d383cab-e43a-46d7-a3d8-c4b281959210 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:39.975353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d383cab-e43a-46d7-a3d8-c4b281959210 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:39.976113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d383cab-e43a-46d7-a3d8-c4b281959210 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:39.976548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d383cab-e43a-46d7-a3d8-c4b281959210 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:40.035796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1d383cab-e43a-46d7-a3d8-c4b281959210 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:40.044046 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1d383cab-e43a-46d7-a3d8-c4b281959210 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:40.045588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1d383cab-e43a-46d7-a3d8-c4b281959210 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:40.046721 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1d383cab-e43a-46d7-a3d8-c4b281959210 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083766 Jun 10 13:24:40.047271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 985/1888] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:39 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:41.065992 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-4dee8f74-4cab-48b2-bd3e-18a6d89d57c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:41.066596 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-4dee8f74-4cab-48b2-bd3e-18a6d89d57c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:41.072731 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4dee8f74-4cab-48b2-bd3e-18a6d89d57c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:41.073199 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4dee8f74-4cab-48b2-bd3e-18a6d89d57c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:41.073947 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4dee8f74-4cab-48b2-bd3e-18a6d89d57c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:41.074378 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-4dee8f74-4cab-48b2-bd3e-18a6d89d57c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:41.133283 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4dee8f74-4cab-48b2-bd3e-18a6d89d57c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:41.141483 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4dee8f74-4cab-48b2-bd3e-18a6d89d57c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:41.143030 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-4dee8f74-4cab-48b2-bd3e-18a6d89d57c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:41.144153 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-4dee8f74-4cab-48b2-bd3e-18a6d89d57c3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082688 Jun 10 13:24:41.144698 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 904/1889] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:41 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:42.162371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6e43658a-9e28-4f71-8c9e-808c74786f71 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:42.162976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6e43658a-9e28-4f71-8c9e-808c74786f71 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:42.169514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e43658a-9e28-4f71-8c9e-808c74786f71 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:42.169963 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e43658a-9e28-4f71-8c9e-808c74786f71 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:42.170753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e43658a-9e28-4f71-8c9e-808c74786f71 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:42.171188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e43658a-9e28-4f71-8c9e-808c74786f71 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:42.230450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6e43658a-9e28-4f71-8c9e-808c74786f71 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:42.239240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6e43658a-9e28-4f71-8c9e-808c74786f71 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:42.240738 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6e43658a-9e28-4f71-8c9e-808c74786f71 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:42.241901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6e43658a-9e28-4f71-8c9e-808c74786f71 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083444 Jun 10 13:24:42.242450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 986/1890] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:42 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:43.259632 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-6da3526d-466c-4763-a83d-7ec559bac5ac tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:43.260162 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-6da3526d-466c-4763-a83d-7ec559bac5ac tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:43.267411 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6da3526d-466c-4763-a83d-7ec559bac5ac tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:43.267858 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6da3526d-466c-4763-a83d-7ec559bac5ac tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:43.268614 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6da3526d-466c-4763-a83d-7ec559bac5ac tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:43.269074 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-6da3526d-466c-4763-a83d-7ec559bac5ac tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:43.328238 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6da3526d-466c-4763-a83d-7ec559bac5ac tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:43.336846 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6da3526d-466c-4763-a83d-7ec559bac5ac tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:43.338330 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-6da3526d-466c-4763-a83d-7ec559bac5ac tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:43.339468 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-6da3526d-466c-4763-a83d-7ec559bac5ac tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083800 Jun 10 13:24:43.340277 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 905/1891] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:43 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:44.358697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7168237c-73fa-41a5-98a4-27d0a7ca86c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:44.359300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7168237c-73fa-41a5-98a4-27d0a7ca86c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:44.366263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7168237c-73fa-41a5-98a4-27d0a7ca86c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:44.366703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7168237c-73fa-41a5-98a4-27d0a7ca86c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:44.367450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7168237c-73fa-41a5-98a4-27d0a7ca86c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:44.367876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7168237c-73fa-41a5-98a4-27d0a7ca86c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:44.431231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7168237c-73fa-41a5-98a4-27d0a7ca86c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:44.439560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7168237c-73fa-41a5-98a4-27d0a7ca86c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:44.441067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7168237c-73fa-41a5-98a4-27d0a7ca86c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:44.442189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7168237c-73fa-41a5-98a4-27d0a7ca86c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087679 Jun 10 13:24:44.442738 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 987/1892] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:44 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:45.460578 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-3ddaa2f7-6b82-4514-9056-6dc5c258ab66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:45.461670 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-3ddaa2f7-6b82-4514-9056-6dc5c258ab66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:45.470368 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ddaa2f7-6b82-4514-9056-6dc5c258ab66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:45.470817 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ddaa2f7-6b82-4514-9056-6dc5c258ab66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:45.471844 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ddaa2f7-6b82-4514-9056-6dc5c258ab66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:45.472278 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-3ddaa2f7-6b82-4514-9056-6dc5c258ab66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:45.535229 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3ddaa2f7-6b82-4514-9056-6dc5c258ab66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:45.543472 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3ddaa2f7-6b82-4514-9056-6dc5c258ab66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:45.544978 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-3ddaa2f7-6b82-4514-9056-6dc5c258ab66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:45.546113 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-3ddaa2f7-6b82-4514-9056-6dc5c258ab66 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.089586 Jun 10 13:24:45.546668 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 906/1893] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:45 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:46.565091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b9ebf8f8-9420-45ab-ac94-4354c614de5b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:46.565630 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b9ebf8f8-9420-45ab-ac94-4354c614de5b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:46.572162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9ebf8f8-9420-45ab-ac94-4354c614de5b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:46.572612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9ebf8f8-9420-45ab-ac94-4354c614de5b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:46.573442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9ebf8f8-9420-45ab-ac94-4354c614de5b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:46.573878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b9ebf8f8-9420-45ab-ac94-4354c614de5b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:46.640917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b9ebf8f8-9420-45ab-ac94-4354c614de5b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:46.646813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b9ebf8f8-9420-45ab-ac94-4354c614de5b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:46.648307 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b9ebf8f8-9420-45ab-ac94-4354c614de5b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:46.649526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b9ebf8f8-9420-45ab-ac94-4354c614de5b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.089021 Jun 10 13:24:46.650093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 988/1894] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:46 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:47.667799 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-c97a27bf-1bd7-4e32-ac43-c4c4f3c3eb94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:47.668307 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-c97a27bf-1bd7-4e32-ac43-c4c4f3c3eb94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:47.674890 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c97a27bf-1bd7-4e32-ac43-c4c4f3c3eb94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:47.675349 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c97a27bf-1bd7-4e32-ac43-c4c4f3c3eb94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:47.676138 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c97a27bf-1bd7-4e32-ac43-c4c4f3c3eb94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:47.676571 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-c97a27bf-1bd7-4e32-ac43-c4c4f3c3eb94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:47.736531 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c97a27bf-1bd7-4e32-ac43-c4c4f3c3eb94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:47.744789 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c97a27bf-1bd7-4e32-ac43-c4c4f3c3eb94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:47.746325 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-c97a27bf-1bd7-4e32-ac43-c4c4f3c3eb94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:47.747460 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-c97a27bf-1bd7-4e32-ac43-c4c4f3c3eb94 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083665 Jun 10 13:24:47.748016 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 907/1895] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:47 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:48.765637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-210d49e5-9adc-4220-b7b2-44953adfbb4d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:48.766264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-210d49e5-9adc-4220-b7b2-44953adfbb4d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:48.768552 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=28 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:24:48.773724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-210d49e5-9adc-4220-b7b2-44953adfbb4d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:48.777305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-210d49e5-9adc-4220-b7b2-44953adfbb4d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:48.777305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-210d49e5-9adc-4220-b7b2-44953adfbb4d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:48.777305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-210d49e5-9adc-4220-b7b2-44953adfbb4d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:48.834875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-210d49e5-9adc-4220-b7b2-44953adfbb4d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:48.843102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-210d49e5-9adc-4220-b7b2-44953adfbb4d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:48.844582 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-210d49e5-9adc-4220-b7b2-44953adfbb4d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:48.845750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-210d49e5-9adc-4220-b7b2-44953adfbb4d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084153 Jun 10 13:24:48.846594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 989/1896] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:48 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:49.863998 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d69933d8-ab8d-4a8b-b6de-4381b713eee3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:49.864603 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d69933d8-ab8d-4a8b-b6de-4381b713eee3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:49.871169 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d69933d8-ab8d-4a8b-b6de-4381b713eee3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:49.871616 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d69933d8-ab8d-4a8b-b6de-4381b713eee3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:49.872377 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d69933d8-ab8d-4a8b-b6de-4381b713eee3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:49.872839 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d69933d8-ab8d-4a8b-b6de-4381b713eee3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:49.876656 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_cell1:SELECT=84 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:24:49.932987 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d69933d8-ab8d-4a8b-b6de-4381b713eee3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:49.941419 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d69933d8-ab8d-4a8b-b6de-4381b713eee3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:49.942909 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d69933d8-ab8d-4a8b-b6de-4381b713eee3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:49.944047 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d69933d8-ab8d-4a8b-b6de-4381b713eee3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084536 Jun 10 13:24:49.944595 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 908/1897] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:49 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:50.961842 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-183493a1-4a1c-4ddd-9f9a-e97cd8111634 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:50.962444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-183493a1-4a1c-4ddd-9f9a-e97cd8111634 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:50.968896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-183493a1-4a1c-4ddd-9f9a-e97cd8111634 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:50.969386 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-183493a1-4a1c-4ddd-9f9a-e97cd8111634 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:50.970140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-183493a1-4a1c-4ddd-9f9a-e97cd8111634 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:50.970569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-183493a1-4a1c-4ddd-9f9a-e97cd8111634 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:51.030930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-183493a1-4a1c-4ddd-9f9a-e97cd8111634 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:51.039251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-183493a1-4a1c-4ddd-9f9a-e97cd8111634 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:51.040729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-183493a1-4a1c-4ddd-9f9a-e97cd8111634 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:51.041875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-183493a1-4a1c-4ddd-9f9a-e97cd8111634 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084023 Jun 10 13:24:51.042453 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 990/1898] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:50 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:52.060275 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-d0e63b67-094a-4de6-a167-519f34eb07ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:52.060986 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-d0e63b67-094a-4de6-a167-519f34eb07ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:52.063808 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG dbcounter [-] [118093] Writing DB stats nova_api:SELECT=28 {{(pid=118093) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:24:52.068633 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d0e63b67-094a-4de6-a167-519f34eb07ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:52.069112 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d0e63b67-094a-4de6-a167-519f34eb07ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:52.069876 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d0e63b67-094a-4de6-a167-519f34eb07ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:52.070314 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-d0e63b67-094a-4de6-a167-519f34eb07ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:52.130310 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d0e63b67-094a-4de6-a167-519f34eb07ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:52.138677 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d0e63b67-094a-4de6-a167-519f34eb07ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:52.140176 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-d0e63b67-094a-4de6-a167-519f34eb07ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:52.141942 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-d0e63b67-094a-4de6-a167-519f34eb07ed tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084998 Jun 10 13:24:52.142503 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 909/1899] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:52 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:53.160189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-10032d2a-a8fc-42fb-891d-d321b3166042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:53.160813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-10032d2a-a8fc-42fb-891d-d321b3166042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:53.167261 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10032d2a-a8fc-42fb-891d-d321b3166042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:53.171578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10032d2a-a8fc-42fb-891d-d321b3166042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:53.171578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10032d2a-a8fc-42fb-891d-d321b3166042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:53.171578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10032d2a-a8fc-42fb-891d-d321b3166042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:53.231260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10032d2a-a8fc-42fb-891d-d321b3166042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:53.239575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10032d2a-a8fc-42fb-891d-d321b3166042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:53.241082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10032d2a-a8fc-42fb-891d-d321b3166042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:53.242204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-10032d2a-a8fc-42fb-891d-d321b3166042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086537 Jun 10 13:24:53.242780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 991/1900] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:53 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:54.259463 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-eeec09b7-e84b-420d-bb42-719a9d244e27 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:54.260070 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-eeec09b7-e84b-420d-bb42-719a9d244e27 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:54.266555 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eeec09b7-e84b-420d-bb42-719a9d244e27 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:54.267005 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eeec09b7-e84b-420d-bb42-719a9d244e27 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:54.267765 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eeec09b7-e84b-420d-bb42-719a9d244e27 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:54.268202 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-eeec09b7-e84b-420d-bb42-719a9d244e27 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:54.331602 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-eeec09b7-e84b-420d-bb42-719a9d244e27 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:54.340218 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-eeec09b7-e84b-420d-bb42-719a9d244e27 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:54.341719 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-eeec09b7-e84b-420d-bb42-719a9d244e27 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:54.342850 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-eeec09b7-e84b-420d-bb42-719a9d244e27 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087366 Jun 10 13:24:54.343663 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 910/1901] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:54 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:55.361641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2baaa551-6897-4980-85d8-0f19d754b8af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:55.362241 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2baaa551-6897-4980-85d8-0f19d754b8af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:55.368633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2baaa551-6897-4980-85d8-0f19d754b8af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:55.369107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2baaa551-6897-4980-85d8-0f19d754b8af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:55.369861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2baaa551-6897-4980-85d8-0f19d754b8af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:55.370286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2baaa551-6897-4980-85d8-0f19d754b8af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:55.429165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2baaa551-6897-4980-85d8-0f19d754b8af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:55.437288 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2baaa551-6897-4980-85d8-0f19d754b8af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:55.438773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2baaa551-6897-4980-85d8-0f19d754b8af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:55.439898 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2baaa551-6897-4980-85d8-0f19d754b8af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082266 Jun 10 13:24:55.440449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 992/1902] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:55 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:56.457750 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-309f897d-facc-4a08-86e3-46293dee58f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:56.458297 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-309f897d-facc-4a08-86e3-46293dee58f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:56.464792 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-309f897d-facc-4a08-86e3-46293dee58f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:56.465249 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-309f897d-facc-4a08-86e3-46293dee58f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:56.466014 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-309f897d-facc-4a08-86e3-46293dee58f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:56.466448 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-309f897d-facc-4a08-86e3-46293dee58f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:56.526320 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-309f897d-facc-4a08-86e3-46293dee58f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:56.535093 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-309f897d-facc-4a08-86e3-46293dee58f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:56.536591 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-309f897d-facc-4a08-86e3-46293dee58f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:56.537769 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-309f897d-facc-4a08-86e3-46293dee58f7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084012 Jun 10 13:24:56.538324 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 911/1903] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:56 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:57.555769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-00c4d232-3f03-4804-8d1d-21445d1e72bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:57.556405 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-00c4d232-3f03-4804-8d1d-21445d1e72bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:57.563308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00c4d232-3f03-4804-8d1d-21445d1e72bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:57.563760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00c4d232-3f03-4804-8d1d-21445d1e72bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:57.564525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00c4d232-3f03-4804-8d1d-21445d1e72bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:57.564992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00c4d232-3f03-4804-8d1d-21445d1e72bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:57.629299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-00c4d232-3f03-4804-8d1d-21445d1e72bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:57.637994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-00c4d232-3f03-4804-8d1d-21445d1e72bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:57.639498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-00c4d232-3f03-4804-8d1d-21445d1e72bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:57.642117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-00c4d232-3f03-4804-8d1d-21445d1e72bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.088852 Jun 10 13:24:57.642117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 993/1904] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:57 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:24:58.660846 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-69e53360-7c2b-44dd-bdab-0e21cbdf2618 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:58.661383 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-69e53360-7c2b-44dd-bdab-0e21cbdf2618 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:58.667831 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69e53360-7c2b-44dd-bdab-0e21cbdf2618 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:58.668281 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69e53360-7c2b-44dd-bdab-0e21cbdf2618 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:58.669078 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69e53360-7c2b-44dd-bdab-0e21cbdf2618 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:58.669517 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-69e53360-7c2b-44dd-bdab-0e21cbdf2618 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:58.731686 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-69e53360-7c2b-44dd-bdab-0e21cbdf2618 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:58.740658 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-69e53360-7c2b-44dd-bdab-0e21cbdf2618 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:58.742169 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-69e53360-7c2b-44dd-bdab-0e21cbdf2618 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:58.743313 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-69e53360-7c2b-44dd-bdab-0e21cbdf2618 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087012 Jun 10 13:24:58.743879 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 912/1905] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:58 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:24:59.761307 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2fe42a3d-f2ea-4da2-b270-5c7d4ad033b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:24:59.761981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2fe42a3d-f2ea-4da2-b270-5c7d4ad033b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:24:59.769321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2fe42a3d-f2ea-4da2-b270-5c7d4ad033b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:59.769846 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2fe42a3d-f2ea-4da2-b270-5c7d4ad033b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:59.770628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2fe42a3d-f2ea-4da2-b270-5c7d4ad033b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:24:59.771096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2fe42a3d-f2ea-4da2-b270-5c7d4ad033b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:24:59.830558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2fe42a3d-f2ea-4da2-b270-5c7d4ad033b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:59.838786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2fe42a3d-f2ea-4da2-b270-5c7d4ad033b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:59.840271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2fe42a3d-f2ea-4da2-b270-5c7d4ad033b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:24:59.841443 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2fe42a3d-f2ea-4da2-b270-5c7d4ad033b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084162 Jun 10 13:24:59.841992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 994/1906] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:24:59 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:25:00.859559 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-24fb16c1-e408-410f-af36-8a74864d20fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:25:00.860203 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-24fb16c1-e408-410f-af36-8a74864d20fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:25:00.866687 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-24fb16c1-e408-410f-af36-8a74864d20fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:25:00.867136 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-24fb16c1-e408-410f-af36-8a74864d20fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:25:00.867894 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-24fb16c1-e408-410f-af36-8a74864d20fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:25:00.868329 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-24fb16c1-e408-410f-af36-8a74864d20fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:25:00.926220 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-24fb16c1-e408-410f-af36-8a74864d20fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:25:00.934505 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-24fb16c1-e408-410f-af36-8a74864d20fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:25:00.935997 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-24fb16c1-e408-410f-af36-8a74864d20fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:25:00.937172 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-24fb16c1-e408-410f-af36-8a74864d20fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.081457 Jun 10 13:25:00.937727 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 913/1907] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:25:00 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:25:01.955140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-97d52060-7093-4981-9930-c893c2116e35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:25:01.955744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-97d52060-7093-4981-9930-c893c2116e35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:25:01.961772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97d52060-7093-4981-9930-c893c2116e35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:25:01.962217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97d52060-7093-4981-9930-c893c2116e35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:25:01.962974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97d52060-7093-4981-9930-c893c2116e35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:25:01.963405 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97d52060-7093-4981-9930-c893c2116e35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:25:01.965968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=84 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:25:02.025888 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-97d52060-7093-4981-9930-c893c2116e35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:25:02.035101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-97d52060-7093-4981-9930-c893c2116e35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:25:02.036593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-97d52060-7093-4981-9930-c893c2116e35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:25:02.037773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-97d52060-7093-4981-9930-c893c2116e35 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086748 Jun 10 13:25:02.038332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 995/1908] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:25:01 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:25:03.055709 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-7fa99ba7-5f95-411d-aa90-06c7119bd569 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:25:03.056409 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-7fa99ba7-5f95-411d-aa90-06c7119bd569 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:25:03.062939 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7fa99ba7-5f95-411d-aa90-06c7119bd569 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:25:03.063388 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7fa99ba7-5f95-411d-aa90-06c7119bd569 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:25:03.064736 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7fa99ba7-5f95-411d-aa90-06c7119bd569 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:25:03.065192 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-7fa99ba7-5f95-411d-aa90-06c7119bd569 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:25:03.127523 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7fa99ba7-5f95-411d-aa90-06c7119bd569 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:25:03.135879 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7fa99ba7-5f95-411d-aa90-06c7119bd569 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:25:03.137461 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-7fa99ba7-5f95-411d-aa90-06c7119bd569 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:25:03.138594 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: INFO nova.api.openstack.requestlog [None req-7fa99ba7-5f95-411d-aa90-06c7119bd569 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086821 Jun 10 13:25:03.139160 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: [pid: 118093|app: 0|req: 914/1909] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:25:03 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:25:04.155413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b60900c5-6e9e-4ac0-b5d2-ad911d3311b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:25:04.156002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b60900c5-6e9e-4ac0-b5d2-ad911d3311b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:25:04.163353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b60900c5-6e9e-4ac0-b5d2-ad911d3311b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:25:04.163797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b60900c5-6e9e-4ac0-b5d2-ad911d3311b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:25:04.164557 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b60900c5-6e9e-4ac0-b5d2-ad911d3311b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:25:04.165012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b60900c5-6e9e-4ac0-b5d2-ad911d3311b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:25:04.224287 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b60900c5-6e9e-4ac0-b5d2-ad911d3311b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:25:04.232568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b60900c5-6e9e-4ac0-b5d2-ad911d3311b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:25:04.234101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b60900c5-6e9e-4ac0-b5d2-ad911d3311b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:25:04.235957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b60900c5-6e9e-4ac0-b5d2-ad911d3311b2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083828 Jun 10 13:25:04.236519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 996/1910] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:25:04 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:25:05.253590 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.api.openstack.wsgi [None req-a37d0b61-de11-4c53-b784-c7190790fd75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118093) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:25:05.254192 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.compute.api [None req-a37d0b61-de11-4c53-b784-c7190790fd75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118093) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:25:05.260679 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a37d0b61-de11-4c53-b784-c7190790fd75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:25:05.261151 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a37d0b61-de11-4c53-b784-c7190790fd75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:25:05.261897 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a37d0b61-de11-4c53-b784-c7190790fd75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:25:05.262327 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG oslo_concurrency.lockutils [None req-a37d0b61-de11-4c53-b784-c7190790fd75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118093) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:25:05.321601 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a37d0b61-de11-4c53-b784-c7190790fd75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:25:05.330423 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: DEBUG nova.policy [None req-a37d0b61-de11-4c53-b784-c7190790fd75 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118093) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:25:05.351428 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: Traceback (most recent call last): Jun 10 13:25:05.351578 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers Jun 10 13:25:05.351578 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: timer() Jun 10 13:25:05.351578 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ Jun 10 13:25:05.351578 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: cb(*args, **kw) Jun 10 13:25:05.351578 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: File "/usr/local/lib/python3.8/dist-packages/eventlet/semaphore.py", line 147, in _do_acquire Jun 10 13:25:05.351578 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: waiter.switch() Jun 10 13:25:05.351578 dv-cinder-master-3-iscsi devstack@n-api.service[118093]: greenlet.error: cannot switch to a different thread Jun 10 13:25:20.618517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-617758ae-1716-4da8-b9df-51a4a6366772 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:25:20.619068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-617758ae-1716-4da8-b9df-51a4a6366772 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:25:20.621863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=15 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:25:20.622524 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=11 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:25:20.628379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-617758ae-1716-4da8-b9df-51a4a6366772 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:25:20.628379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-617758ae-1716-4da8-b9df-51a4a6366772 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:25:20.628974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-617758ae-1716-4da8-b9df-51a4a6366772 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:25:20.629585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-617758ae-1716-4da8-b9df-51a4a6366772 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:25:20.646372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-617758ae-1716-4da8-b9df-51a4a6366772 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] HTTP exception thrown: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-617758ae-1716-4da8-b9df-51a4a6366772 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Request method failure captured: Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 HTTP/1.1 Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0GLIJPQ8lSWSCxXU9neIcS9fWNdW4XJF-omQVn8xvgfajtrPtR9xT7yJRQ7uDkAeVSjjX8Zr-TojYB67_YvghXuxNAaWo9VFXLD41gtOICuurNBjdxNv9Y58tQmvsCSdX2FomGdBsssQdaFXDYnlp0d7vxIpJLVub7cC52vynSBvwE Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 6dace38cecd44f87becbf6dc213851b3 Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:25:20.651320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:25:20.653461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_6dace38cecd44f87becbf6dc213851b3", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:25:20.653461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:25:20.653461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 6dace38cecd44f87becbf6dc213851b3 Jun 10 13:25:20.653461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:25:20.653461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeTestJSON-1599090335-project Jun 10 13:25:20.653461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:25:20.653461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:25:20.653461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 9c8b2c27b69a4776ab6b7b7820da02d9 Jun 10 13:25:20.653461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeTestJSON-1599090335-project Jun 10 13:25:20.653461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:25:20.653461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:25:20.653461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '6273d596-6d75-49f6-9c03-17ea7b4422d7'} Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:25:20.654364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:25:20.655754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:25:20.658574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-617758ae-1716-4da8-b9df-51a4a6366772 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Returning 404 to user: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:25:20.658574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-617758ae-1716-4da8-b9df-51a4a6366772 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "DELETE /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 404 len: 111 microversion: 2.1 time: 0.034725 Jun 10 13:25:20.658574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 997/1911] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:25:20 2022] DELETE /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:25:20.658574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0eebaa11-5d6b-4840-9b6b-6f91b3ea8f0d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:25:20.659049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0eebaa11-5d6b-4840-9b6b-6f91b3ea8f0d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 6273d596-6d75-49f6-9c03-17ea7b4422d7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:25:20.665399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0eebaa11-5d6b-4840-9b6b-6f91b3ea8f0d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:25:20.665840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0eebaa11-5d6b-4840-9b6b-6f91b3ea8f0d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:25:20.666602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0eebaa11-5d6b-4840-9b6b-6f91b3ea8f0d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:25:20.667030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0eebaa11-5d6b-4840-9b6b-6f91b3ea8f0d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:25:20.681562 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-0eebaa11-5d6b-4840-9b6b-6f91b3ea8f0d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] HTTP exception thrown: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0eebaa11-5d6b-4840-9b6b-6f91b3ea8f0d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Request method failure captured: Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 HTTP/1.1 Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0GLIJPQ8lSWSCxXU9neIcS9fWNdW4XJF-omQVn8xvgfajtrPtR9xT7yJRQ7uDkAeVSjjX8Zr-TojYB67_YvghXuxNAaWo9VFXLD41gtOICuurNBjdxNv9Y58tQmvsCSdX2FomGdBsssQdaFXDYnlp0d7vxIpJLVub7cC52vynSBvwE Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 6dace38cecd44f87becbf6dc213851b3 Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:25:20.688164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:25:20.689956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_6dace38cecd44f87becbf6dc213851b3", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:25:20.689956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:25:20.689956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 6dace38cecd44f87becbf6dc213851b3 Jun 10 13:25:20.689956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:25:20.689956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeTestJSON-1599090335-project Jun 10 13:25:20.689956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:25:20.689956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:25:20.689956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 9c8b2c27b69a4776ab6b7b7820da02d9 Jun 10 13:25:20.689956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeTestJSON-1599090335-project Jun 10 13:25:20.689956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:25:20.689956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:25:20.689956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '6273d596-6d75-49f6-9c03-17ea7b4422d7'} Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:25:20.690904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. Jun 10 13:25:20.692322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:25:20.693751 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0eebaa11-5d6b-4840-9b6b-6f91b3ea8f0d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Returning 404 to user: Instance 6273d596-6d75-49f6-9c03-17ea7b4422d7 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:25:20.693751 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0eebaa11-5d6b-4840-9b6b-6f91b3ea8f0d tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7" status: 404 len: 111 microversion: 2.1 time: 0.030617 Jun 10 13:25:20.693751 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 998/1912] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:25:20 2022] GET /compute/v2.1/servers/6273d596-6d75-49f6-9c03-17ea7b4422d7 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:25:35.233861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:25:35.234564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:25:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:26:05.315416 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d97f577f-60e4-48e8-8f4f-ede6387f5152 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:05.316134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d97f577f-60e4-48e8-8f4f-ede6387f5152 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:05.318364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=4 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:26:05.323413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d97f577f-60e4-48e8-8f4f-ede6387f5152 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:05.323866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d97f577f-60e4-48e8-8f4f-ede6387f5152 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:05.324634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d97f577f-60e4-48e8-8f4f-ede6387f5152 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:05.325139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d97f577f-60e4-48e8-8f4f-ede6387f5152 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:05.536204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d97f577f-60e4-48e8-8f4f-ede6387f5152 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:05.544626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d97f577f-60e4-48e8-8f4f-ede6387f5152 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:05.546171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d97f577f-60e4-48e8-8f4f-ede6387f5152 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:05.547321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d97f577f-60e4-48e8-8f4f-ede6387f5152 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.236635 Jun 10 13:26:05.548496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 999/1913] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:05 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:06.565665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bfaa2070-cc2b-40cc-b7b4-71e385cb1c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:06.566272 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bfaa2070-cc2b-40cc-b7b4-71e385cb1c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:06.572798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bfaa2070-cc2b-40cc-b7b4-71e385cb1c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:06.573245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bfaa2070-cc2b-40cc-b7b4-71e385cb1c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:06.574004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bfaa2070-cc2b-40cc-b7b4-71e385cb1c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:06.574445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bfaa2070-cc2b-40cc-b7b4-71e385cb1c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:06.633278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bfaa2070-cc2b-40cc-b7b4-71e385cb1c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:06.642064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bfaa2070-cc2b-40cc-b7b4-71e385cb1c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:06.643560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bfaa2070-cc2b-40cc-b7b4-71e385cb1c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:06.644692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bfaa2070-cc2b-40cc-b7b4-71e385cb1c0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082920 Jun 10 13:26:06.645267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1000/1914] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:06 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:26:07.662558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-17331535-5246-4bbd-a4a5-3360924e600b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:07.663180 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-17331535-5246-4bbd-a4a5-3360924e600b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:07.669534 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17331535-5246-4bbd-a4a5-3360924e600b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:07.669972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17331535-5246-4bbd-a4a5-3360924e600b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:07.670724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17331535-5246-4bbd-a4a5-3360924e600b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:07.671151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17331535-5246-4bbd-a4a5-3360924e600b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:07.731590 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-17331535-5246-4bbd-a4a5-3360924e600b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:07.740512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-17331535-5246-4bbd-a4a5-3360924e600b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:07.742028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-17331535-5246-4bbd-a4a5-3360924e600b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:07.743160 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-17331535-5246-4bbd-a4a5-3360924e600b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084675 Jun 10 13:26:07.743704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1001/1915] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:07 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:08.761042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-be51b28d-cc53-4966-8941-d6fa808fbc55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:08.761696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-be51b28d-cc53-4966-8941-d6fa808fbc55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:08.768089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be51b28d-cc53-4966-8941-d6fa808fbc55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:08.770170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be51b28d-cc53-4966-8941-d6fa808fbc55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:08.770170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be51b28d-cc53-4966-8941-d6fa808fbc55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:08.770170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be51b28d-cc53-4966-8941-d6fa808fbc55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:08.829497 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-be51b28d-cc53-4966-8941-d6fa808fbc55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:08.837818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-be51b28d-cc53-4966-8941-d6fa808fbc55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:08.839331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-be51b28d-cc53-4966-8941-d6fa808fbc55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:08.840521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-be51b28d-cc53-4966-8941-d6fa808fbc55 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083494 Jun 10 13:26:08.841114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1002/1916] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:08 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:09.858651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c3ec9e71-ce0d-4352-9577-6e55e11b6309 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:09.859367 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c3ec9e71-ce0d-4352-9577-6e55e11b6309 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:09.866375 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3ec9e71-ce0d-4352-9577-6e55e11b6309 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:09.866821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3ec9e71-ce0d-4352-9577-6e55e11b6309 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:09.867572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3ec9e71-ce0d-4352-9577-6e55e11b6309 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:09.868000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3ec9e71-ce0d-4352-9577-6e55e11b6309 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:09.932224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c3ec9e71-ce0d-4352-9577-6e55e11b6309 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:09.940666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c3ec9e71-ce0d-4352-9577-6e55e11b6309 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:09.942195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c3ec9e71-ce0d-4352-9577-6e55e11b6309 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:09.943331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c3ec9e71-ce0d-4352-9577-6e55e11b6309 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.088695 Jun 10 13:26:09.943878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1003/1917] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:09 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:10.961006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dab3aab8-d8d4-4bd6-8927-a00fa8ad906a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:10.961681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dab3aab8-d8d4-4bd6-8927-a00fa8ad906a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:10.968608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dab3aab8-d8d4-4bd6-8927-a00fa8ad906a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:10.969626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dab3aab8-d8d4-4bd6-8927-a00fa8ad906a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:10.970393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dab3aab8-d8d4-4bd6-8927-a00fa8ad906a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:10.970826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dab3aab8-d8d4-4bd6-8927-a00fa8ad906a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:11.030662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dab3aab8-d8d4-4bd6-8927-a00fa8ad906a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:11.039525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dab3aab8-d8d4-4bd6-8927-a00fa8ad906a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:11.041044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dab3aab8-d8d4-4bd6-8927-a00fa8ad906a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:11.042174 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dab3aab8-d8d4-4bd6-8927-a00fa8ad906a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085128 Jun 10 13:26:11.042727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1004/1918] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:10 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:12.060313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3041f145-1289-4ed1-acff-17d96070f897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:12.061010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3041f145-1289-4ed1-acff-17d96070f897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:12.067984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3041f145-1289-4ed1-acff-17d96070f897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:12.068431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3041f145-1289-4ed1-acff-17d96070f897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:12.069233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3041f145-1289-4ed1-acff-17d96070f897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:12.069666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3041f145-1289-4ed1-acff-17d96070f897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:12.129154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3041f145-1289-4ed1-acff-17d96070f897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:12.137381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3041f145-1289-4ed1-acff-17d96070f897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:12.138878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3041f145-1289-4ed1-acff-17d96070f897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:12.140009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3041f145-1289-4ed1-acff-17d96070f897 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083663 Jun 10 13:26:12.140558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1005/1919] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:12 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:13.157107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d52ec9af-7407-4ba2-a059-2b404aaef114 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:13.157794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d52ec9af-7407-4ba2-a059-2b404aaef114 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:13.164047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d52ec9af-7407-4ba2-a059-2b404aaef114 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:13.164496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d52ec9af-7407-4ba2-a059-2b404aaef114 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:13.165289 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d52ec9af-7407-4ba2-a059-2b404aaef114 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:13.165722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d52ec9af-7407-4ba2-a059-2b404aaef114 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:13.224401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d52ec9af-7407-4ba2-a059-2b404aaef114 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:13.232578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d52ec9af-7407-4ba2-a059-2b404aaef114 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:13.234131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d52ec9af-7407-4ba2-a059-2b404aaef114 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:13.235777 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d52ec9af-7407-4ba2-a059-2b404aaef114 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082650 Jun 10 13:26:13.236331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1006/1920] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:13 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:14.254517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-545f62dc-d31b-447d-8cd0-544289bd4f0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:14.255196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-545f62dc-d31b-447d-8cd0-544289bd4f0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:14.261853 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-545f62dc-d31b-447d-8cd0-544289bd4f0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:14.262294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-545f62dc-d31b-447d-8cd0-544289bd4f0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:14.263051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-545f62dc-d31b-447d-8cd0-544289bd4f0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:14.263481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-545f62dc-d31b-447d-8cd0-544289bd4f0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:14.331461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-545f62dc-d31b-447d-8cd0-544289bd4f0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:14.343319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-545f62dc-d31b-447d-8cd0-544289bd4f0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:14.343319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-545f62dc-d31b-447d-8cd0-544289bd4f0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:14.343319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-545f62dc-d31b-447d-8cd0-544289bd4f0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.091941 Jun 10 13:26:14.343692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1007/1921] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:14 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:15.360867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ca5b32e3-dc1d-4949-b690-3cbcf47b1858 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:15.361470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ca5b32e3-dc1d-4949-b690-3cbcf47b1858 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:15.367971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca5b32e3-dc1d-4949-b690-3cbcf47b1858 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:15.368412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca5b32e3-dc1d-4949-b690-3cbcf47b1858 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:15.369184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca5b32e3-dc1d-4949-b690-3cbcf47b1858 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:15.369613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca5b32e3-dc1d-4949-b690-3cbcf47b1858 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:15.428915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ca5b32e3-dc1d-4949-b690-3cbcf47b1858 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:15.442618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ca5b32e3-dc1d-4949-b690-3cbcf47b1858 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:15.444099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ca5b32e3-dc1d-4949-b690-3cbcf47b1858 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:15.445274 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ca5b32e3-dc1d-4949-b690-3cbcf47b1858 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.088404 Jun 10 13:26:15.445830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1008/1922] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:15 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:16.463893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5d7666d3-a7ff-438b-987d-d933a056bbfe tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:16.464501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5d7666d3-a7ff-438b-987d-d933a056bbfe tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:16.471951 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d7666d3-a7ff-438b-987d-d933a056bbfe tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:16.472399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d7666d3-a7ff-438b-987d-d933a056bbfe tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:16.473602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d7666d3-a7ff-438b-987d-d933a056bbfe tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:16.474309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5d7666d3-a7ff-438b-987d-d933a056bbfe tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:16.547672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5d7666d3-a7ff-438b-987d-d933a056bbfe tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:16.556714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5d7666d3-a7ff-438b-987d-d933a056bbfe tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:16.558414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5d7666d3-a7ff-438b-987d-d933a056bbfe tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:16.559598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5d7666d3-a7ff-438b-987d-d933a056bbfe tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.099683 Jun 10 13:26:16.560204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1009/1923] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:16 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:17.577976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-848351e4-2e3f-4d25-9c2d-827b70069b68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:17.582337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-848351e4-2e3f-4d25-9c2d-827b70069b68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:17.585041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-848351e4-2e3f-4d25-9c2d-827b70069b68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:17.585483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-848351e4-2e3f-4d25-9c2d-827b70069b68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:17.586293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-848351e4-2e3f-4d25-9c2d-827b70069b68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:17.586668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-848351e4-2e3f-4d25-9c2d-827b70069b68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:17.651209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-848351e4-2e3f-4d25-9c2d-827b70069b68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:17.660358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-848351e4-2e3f-4d25-9c2d-827b70069b68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:17.661883 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-848351e4-2e3f-4d25-9c2d-827b70069b68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:17.663007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-848351e4-2e3f-4d25-9c2d-827b70069b68 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.088993 Jun 10 13:26:17.663560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1010/1924] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:17 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:26:18.684676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-288e5cf0-e84d-479c-a1f4-0ff307f83d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:18.685356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-288e5cf0-e84d-479c-a1f4-0ff307f83d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:18.691662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-288e5cf0-e84d-479c-a1f4-0ff307f83d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:18.692106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-288e5cf0-e84d-479c-a1f4-0ff307f83d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:18.692887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-288e5cf0-e84d-479c-a1f4-0ff307f83d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:18.693316 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-288e5cf0-e84d-479c-a1f4-0ff307f83d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:18.761390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-288e5cf0-e84d-479c-a1f4-0ff307f83d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:18.770261 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-288e5cf0-e84d-479c-a1f4-0ff307f83d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:18.771924 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-288e5cf0-e84d-479c-a1f4-0ff307f83d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:18.773102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-288e5cf0-e84d-479c-a1f4-0ff307f83d2e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.092323 Jun 10 13:26:18.773664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1011/1925] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:18 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:26:19.792165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9a2fd3dd-e1e6-4a63-914c-7ff0829be0df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:19.792854 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9a2fd3dd-e1e6-4a63-914c-7ff0829be0df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:19.799768 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a2fd3dd-e1e6-4a63-914c-7ff0829be0df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:19.800153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a2fd3dd-e1e6-4a63-914c-7ff0829be0df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:19.800944 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a2fd3dd-e1e6-4a63-914c-7ff0829be0df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:19.801375 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a2fd3dd-e1e6-4a63-914c-7ff0829be0df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:19.860026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9a2fd3dd-e1e6-4a63-914c-7ff0829be0df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:19.869289 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9a2fd3dd-e1e6-4a63-914c-7ff0829be0df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:19.870807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9a2fd3dd-e1e6-4a63-914c-7ff0829be0df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:19.871955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9a2fd3dd-e1e6-4a63-914c-7ff0829be0df tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083685 Jun 10 13:26:19.872506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1012/1926] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:19 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:20.890252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-04601507-2594-48de-9841-714e7a63f6c1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:20.890906 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-04601507-2594-48de-9841-714e7a63f6c1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:20.897846 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04601507-2594-48de-9841-714e7a63f6c1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:20.898291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04601507-2594-48de-9841-714e7a63f6c1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:20.899047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04601507-2594-48de-9841-714e7a63f6c1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:20.899478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04601507-2594-48de-9841-714e7a63f6c1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:20.958127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-04601507-2594-48de-9841-714e7a63f6c1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:20.966467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-04601507-2594-48de-9841-714e7a63f6c1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:20.968257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-04601507-2594-48de-9841-714e7a63f6c1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:20.969434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-04601507-2594-48de-9841-714e7a63f6c1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083123 Jun 10 13:26:20.970627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1013/1927] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:20 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:21.987785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0437c516-6cf6-4352-93f4-8e2cbd3b6f1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:21.988445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0437c516-6cf6-4352-93f4-8e2cbd3b6f1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:21.994857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0437c516-6cf6-4352-93f4-8e2cbd3b6f1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:21.995299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0437c516-6cf6-4352-93f4-8e2cbd3b6f1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:21.996044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0437c516-6cf6-4352-93f4-8e2cbd3b6f1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:21.996469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0437c516-6cf6-4352-93f4-8e2cbd3b6f1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:22.055170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0437c516-6cf6-4352-93f4-8e2cbd3b6f1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:22.058476 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:26:22.064148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0437c516-6cf6-4352-93f4-8e2cbd3b6f1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:22.065661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0437c516-6cf6-4352-93f4-8e2cbd3b6f1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:22.066779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0437c516-6cf6-4352-93f4-8e2cbd3b6f1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082917 Jun 10 13:26:22.067334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1014/1928] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:21 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:23.085308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6727c46c-b650-4878-844b-c3a1de29379c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:23.086095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6727c46c-b650-4878-844b-c3a1de29379c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:23.092662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6727c46c-b650-4878-844b-c3a1de29379c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:23.093387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6727c46c-b650-4878-844b-c3a1de29379c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:23.094139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6727c46c-b650-4878-844b-c3a1de29379c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:23.094564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6727c46c-b650-4878-844b-c3a1de29379c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:23.157807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6727c46c-b650-4878-844b-c3a1de29379c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:23.166198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6727c46c-b650-4878-844b-c3a1de29379c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:23.167710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6727c46c-b650-4878-844b-c3a1de29379c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:23.168884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6727c46c-b650-4878-844b-c3a1de29379c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087480 Jun 10 13:26:23.169439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1015/1929] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:23 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:26:24.186817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ce38e38d-d3ce-46e9-aa61-bb2916387f03 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:24.187684 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ce38e38d-d3ce-46e9-aa61-bb2916387f03 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:24.194224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ce38e38d-d3ce-46e9-aa61-bb2916387f03 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:24.194678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ce38e38d-d3ce-46e9-aa61-bb2916387f03 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:24.195466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ce38e38d-d3ce-46e9-aa61-bb2916387f03 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:24.195895 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ce38e38d-d3ce-46e9-aa61-bb2916387f03 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:24.260617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ce38e38d-d3ce-46e9-aa61-bb2916387f03 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:24.269563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ce38e38d-d3ce-46e9-aa61-bb2916387f03 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:24.270573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ce38e38d-d3ce-46e9-aa61-bb2916387f03 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:24.271716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ce38e38d-d3ce-46e9-aa61-bb2916387f03 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.088884 Jun 10 13:26:24.272265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1016/1930] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:24 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:24.372724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:26:24.372872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:26:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:26:25.289057 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4dfdfc2a-f3f9-4fd7-96e9-df88d908a7d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:25.289728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4dfdfc2a-f3f9-4fd7-96e9-df88d908a7d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:25.296062 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4dfdfc2a-f3f9-4fd7-96e9-df88d908a7d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:25.296499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4dfdfc2a-f3f9-4fd7-96e9-df88d908a7d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:25.297273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4dfdfc2a-f3f9-4fd7-96e9-df88d908a7d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:25.298200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4dfdfc2a-f3f9-4fd7-96e9-df88d908a7d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:25.360940 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4dfdfc2a-f3f9-4fd7-96e9-df88d908a7d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:25.369304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4dfdfc2a-f3f9-4fd7-96e9-df88d908a7d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:25.370782 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4dfdfc2a-f3f9-4fd7-96e9-df88d908a7d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:25.371907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4dfdfc2a-f3f9-4fd7-96e9-df88d908a7d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086805 Jun 10 13:26:25.372490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1017/1931] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:25 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:26.390234 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-78115e6f-6537-436a-b868-746b5674a2f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:26.390815 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-78115e6f-6537-436a-b868-746b5674a2f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:26.397884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78115e6f-6537-436a-b868-746b5674a2f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:26.398321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78115e6f-6537-436a-b868-746b5674a2f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:26.399069 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78115e6f-6537-436a-b868-746b5674a2f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:26.399506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78115e6f-6537-436a-b868-746b5674a2f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:26.460058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-78115e6f-6537-436a-b868-746b5674a2f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:26.468827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-78115e6f-6537-436a-b868-746b5674a2f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:26.470358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-78115e6f-6537-436a-b868-746b5674a2f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:26.471523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-78115e6f-6537-436a-b868-746b5674a2f8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085180 Jun 10 13:26:26.472082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1018/1932] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:26 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:27.488513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6228bb94-286b-4544-a9c5-1989cbd8cdd3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:27.489140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6228bb94-286b-4544-a9c5-1989cbd8cdd3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:27.495539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6228bb94-286b-4544-a9c5-1989cbd8cdd3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:27.495976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6228bb94-286b-4544-a9c5-1989cbd8cdd3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:27.496719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6228bb94-286b-4544-a9c5-1989cbd8cdd3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:27.497173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6228bb94-286b-4544-a9c5-1989cbd8cdd3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:27.558540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6228bb94-286b-4544-a9c5-1989cbd8cdd3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:27.567425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6228bb94-286b-4544-a9c5-1989cbd8cdd3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:27.568939 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6228bb94-286b-4544-a9c5-1989cbd8cdd3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:27.570063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6228bb94-286b-4544-a9c5-1989cbd8cdd3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085506 Jun 10 13:26:27.570613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1019/1933] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:27 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:28.586867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-36b95d60-adcb-497c-a89c-f78324c4c4bb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:28.587424 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-36b95d60-adcb-497c-a89c-f78324c4c4bb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:28.594156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36b95d60-adcb-497c-a89c-f78324c4c4bb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:28.595095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36b95d60-adcb-497c-a89c-f78324c4c4bb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:28.595839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36b95d60-adcb-497c-a89c-f78324c4c4bb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:28.596268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36b95d60-adcb-497c-a89c-f78324c4c4bb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:28.654709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-36b95d60-adcb-497c-a89c-f78324c4c4bb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:28.663317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-36b95d60-adcb-497c-a89c-f78324c4c4bb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:28.664817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-36b95d60-adcb-497c-a89c-f78324c4c4bb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:28.665939 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-36b95d60-adcb-497c-a89c-f78324c4c4bb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083078 Jun 10 13:26:28.666496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1020/1934] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:28 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:29.684445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ec9976f0-38e7-4395-86b5-a116435365a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:29.685015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ec9976f0-38e7-4395-86b5-a116435365a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:29.691994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ec9976f0-38e7-4395-86b5-a116435365a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:29.692442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ec9976f0-38e7-4395-86b5-a116435365a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:29.693327 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ec9976f0-38e7-4395-86b5-a116435365a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:29.693779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ec9976f0-38e7-4395-86b5-a116435365a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:29.752975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ec9976f0-38e7-4395-86b5-a116435365a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:29.761251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ec9976f0-38e7-4395-86b5-a116435365a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:29.762736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ec9976f0-38e7-4395-86b5-a116435365a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:29.763862 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ec9976f0-38e7-4395-86b5-a116435365a2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083677 Jun 10 13:26:29.764412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1021/1935] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:29 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:30.782407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-92a58d51-cdab-433f-9337-865d60b32a69 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:30.783076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-92a58d51-cdab-433f-9337-865d60b32a69 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:30.789387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-92a58d51-cdab-433f-9337-865d60b32a69 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:30.789827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-92a58d51-cdab-433f-9337-865d60b32a69 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:30.790575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-92a58d51-cdab-433f-9337-865d60b32a69 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:30.791011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-92a58d51-cdab-433f-9337-865d60b32a69 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:30.849862 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-92a58d51-cdab-433f-9337-865d60b32a69 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:30.858188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-92a58d51-cdab-433f-9337-865d60b32a69 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:30.859678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-92a58d51-cdab-433f-9337-865d60b32a69 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:30.861476 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-92a58d51-cdab-433f-9337-865d60b32a69 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083759 Jun 10 13:26:30.862030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1022/1936] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:30 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:31.879979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8e80bbc9-45d5-4509-a39c-1220e663a451 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:31.880581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8e80bbc9-45d5-4509-a39c-1220e663a451 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:31.886666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8e80bbc9-45d5-4509-a39c-1220e663a451 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:31.887109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8e80bbc9-45d5-4509-a39c-1220e663a451 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:31.887896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8e80bbc9-45d5-4509-a39c-1220e663a451 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:31.888328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8e80bbc9-45d5-4509-a39c-1220e663a451 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:31.957926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8e80bbc9-45d5-4509-a39c-1220e663a451 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:31.966639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8e80bbc9-45d5-4509-a39c-1220e663a451 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:31.972973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8e80bbc9-45d5-4509-a39c-1220e663a451 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:31.972973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8e80bbc9-45d5-4509-a39c-1220e663a451 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.094426 Jun 10 13:26:31.972973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1023/1937] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:31 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:26:32.987425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c0a4f3e0-d781-4dc7-b972-d049d7d6f130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:32.988075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c0a4f3e0-d781-4dc7-b972-d049d7d6f130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:32.994423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c0a4f3e0-d781-4dc7-b972-d049d7d6f130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:32.994860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c0a4f3e0-d781-4dc7-b972-d049d7d6f130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:32.995607 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c0a4f3e0-d781-4dc7-b972-d049d7d6f130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:32.996031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c0a4f3e0-d781-4dc7-b972-d049d7d6f130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:33.056092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c0a4f3e0-d781-4dc7-b972-d049d7d6f130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:33.065295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c0a4f3e0-d781-4dc7-b972-d049d7d6f130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:33.066796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c0a4f3e0-d781-4dc7-b972-d049d7d6f130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:33.067975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c0a4f3e0-d781-4dc7-b972-d049d7d6f130 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084493 Jun 10 13:26:33.068522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1024/1938] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:32 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:34.085704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6ae069be-f931-4e37-af44-949d4caf19e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:34.086416 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6ae069be-f931-4e37-af44-949d4caf19e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:34.092970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ae069be-f931-4e37-af44-949d4caf19e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:34.093409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ae069be-f931-4e37-af44-949d4caf19e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:34.094155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ae069be-f931-4e37-af44-949d4caf19e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:34.094582 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6ae069be-f931-4e37-af44-949d4caf19e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:34.153277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6ae069be-f931-4e37-af44-949d4caf19e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:34.161491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6ae069be-f931-4e37-af44-949d4caf19e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:34.162971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6ae069be-f931-4e37-af44-949d4caf19e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:34.164089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6ae069be-f931-4e37-af44-949d4caf19e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082287 Jun 10 13:26:34.164637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1025/1939] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:34 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:35.182276 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cb634af6-d48b-47e8-a701-21d58b0ea68f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:35.182963 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cb634af6-d48b-47e8-a701-21d58b0ea68f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:35.189274 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb634af6-d48b-47e8-a701-21d58b0ea68f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:35.189717 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb634af6-d48b-47e8-a701-21d58b0ea68f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:35.190470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb634af6-d48b-47e8-a701-21d58b0ea68f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:35.190900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb634af6-d48b-47e8-a701-21d58b0ea68f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:35.250268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cb634af6-d48b-47e8-a701-21d58b0ea68f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:35.259002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cb634af6-d48b-47e8-a701-21d58b0ea68f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:35.260493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cb634af6-d48b-47e8-a701-21d58b0ea68f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:35.262321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cb634af6-d48b-47e8-a701-21d58b0ea68f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083709 Jun 10 13:26:35.263624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1026/1940] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:35 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:36.281025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-42d4ac0b-2935-4b5c-9e66-ef4b395a54d0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:36.282238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-42d4ac0b-2935-4b5c-9e66-ef4b395a54d0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:36.284538 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=57 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:26:36.289257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-42d4ac0b-2935-4b5c-9e66-ef4b395a54d0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:36.289701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-42d4ac0b-2935-4b5c-9e66-ef4b395a54d0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:36.290454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-42d4ac0b-2935-4b5c-9e66-ef4b395a54d0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:36.290902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-42d4ac0b-2935-4b5c-9e66-ef4b395a54d0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:36.353171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-42d4ac0b-2935-4b5c-9e66-ef4b395a54d0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:36.362147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-42d4ac0b-2935-4b5c-9e66-ef4b395a54d0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:36.363622 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-42d4ac0b-2935-4b5c-9e66-ef4b395a54d0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:36.365204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-42d4ac0b-2935-4b5c-9e66-ef4b395a54d0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.088058 Jun 10 13:26:36.365764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1027/1941] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:36 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:37.383179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-10afc4c7-b084-44da-a219-8d4554fa9f98 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:37.383782 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-10afc4c7-b084-44da-a219-8d4554fa9f98 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:37.390247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10afc4c7-b084-44da-a219-8d4554fa9f98 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:37.390694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10afc4c7-b084-44da-a219-8d4554fa9f98 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:37.391443 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10afc4c7-b084-44da-a219-8d4554fa9f98 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:37.391868 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10afc4c7-b084-44da-a219-8d4554fa9f98 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:37.454114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10afc4c7-b084-44da-a219-8d4554fa9f98 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:37.459363 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10afc4c7-b084-44da-a219-8d4554fa9f98 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:37.460895 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10afc4c7-b084-44da-a219-8d4554fa9f98 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:37.462157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-10afc4c7-b084-44da-a219-8d4554fa9f98 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082937 Jun 10 13:26:37.462716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1028/1942] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:37 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:38.480818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b376f057-fa7c-4415-8909-2b1261a0e6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:38.481427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b376f057-fa7c-4415-8909-2b1261a0e6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:38.487843 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b376f057-fa7c-4415-8909-2b1261a0e6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:38.488281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b376f057-fa7c-4415-8909-2b1261a0e6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:38.489055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b376f057-fa7c-4415-8909-2b1261a0e6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:38.489480 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b376f057-fa7c-4415-8909-2b1261a0e6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:38.550419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b376f057-fa7c-4415-8909-2b1261a0e6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:38.558648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b376f057-fa7c-4415-8909-2b1261a0e6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:38.560129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b376f057-fa7c-4415-8909-2b1261a0e6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:38.561286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b376f057-fa7c-4415-8909-2b1261a0e6d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084412 Jun 10 13:26:38.561836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1029/1943] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:38 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:39.579500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e3ee5011-2b95-4ad1-ad79-cdd2e20527d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:39.580216 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e3ee5011-2b95-4ad1-ad79-cdd2e20527d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:39.586696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e3ee5011-2b95-4ad1-ad79-cdd2e20527d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:39.587082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e3ee5011-2b95-4ad1-ad79-cdd2e20527d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:39.587829 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e3ee5011-2b95-4ad1-ad79-cdd2e20527d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:39.588259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e3ee5011-2b95-4ad1-ad79-cdd2e20527d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:39.647473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e3ee5011-2b95-4ad1-ad79-cdd2e20527d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:39.655697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e3ee5011-2b95-4ad1-ad79-cdd2e20527d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:39.657812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e3ee5011-2b95-4ad1-ad79-cdd2e20527d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:39.658938 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e3ee5011-2b95-4ad1-ad79-cdd2e20527d6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083299 Jun 10 13:26:39.659483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1030/1944] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:39 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:40.675967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-84d45dbf-0861-435c-b655-dc5f6a7d661f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:40.676569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-84d45dbf-0861-435c-b655-dc5f6a7d661f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:40.683191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84d45dbf-0861-435c-b655-dc5f6a7d661f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:40.683638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84d45dbf-0861-435c-b655-dc5f6a7d661f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:40.684415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84d45dbf-0861-435c-b655-dc5f6a7d661f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:40.684875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84d45dbf-0861-435c-b655-dc5f6a7d661f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:40.703451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:26:40.748030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-84d45dbf-0861-435c-b655-dc5f6a7d661f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:40.757025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-84d45dbf-0861-435c-b655-dc5f6a7d661f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:40.758525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-84d45dbf-0861-435c-b655-dc5f6a7d661f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:40.759654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-84d45dbf-0861-435c-b655-dc5f6a7d661f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087709 Jun 10 13:26:40.760209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1031/1945] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:40 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:41.778123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-833d66c0-934f-4266-be3b-e3c692a29481 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:41.778789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-833d66c0-934f-4266-be3b-e3c692a29481 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:41.785168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-833d66c0-934f-4266-be3b-e3c692a29481 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:41.785616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-833d66c0-934f-4266-be3b-e3c692a29481 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:41.786402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-833d66c0-934f-4266-be3b-e3c692a29481 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:41.786836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-833d66c0-934f-4266-be3b-e3c692a29481 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:41.849194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-833d66c0-934f-4266-be3b-e3c692a29481 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:41.858612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-833d66c0-934f-4266-be3b-e3c692a29481 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:41.860621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-833d66c0-934f-4266-be3b-e3c692a29481 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:41.861794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-833d66c0-934f-4266-be3b-e3c692a29481 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087651 Jun 10 13:26:41.862356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1032/1946] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:41 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:42.879976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d20eb23a-4f37-441d-8d72-acc7a4dc61d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:42.880613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d20eb23a-4f37-441d-8d72-acc7a4dc61d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:42.887075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d20eb23a-4f37-441d-8d72-acc7a4dc61d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:42.887524 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d20eb23a-4f37-441d-8d72-acc7a4dc61d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:42.888282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d20eb23a-4f37-441d-8d72-acc7a4dc61d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:42.888712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d20eb23a-4f37-441d-8d72-acc7a4dc61d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:42.947697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d20eb23a-4f37-441d-8d72-acc7a4dc61d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:42.956971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d20eb23a-4f37-441d-8d72-acc7a4dc61d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:42.958497 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d20eb23a-4f37-441d-8d72-acc7a4dc61d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:42.959620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d20eb23a-4f37-441d-8d72-acc7a4dc61d4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083597 Jun 10 13:26:42.960169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1033/1947] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:42 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:43.978022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-239cca9f-6b85-4f25-9d1c-9197dc8ad6bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:43.978666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-239cca9f-6b85-4f25-9d1c-9197dc8ad6bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:43.985656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-239cca9f-6b85-4f25-9d1c-9197dc8ad6bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:43.986103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-239cca9f-6b85-4f25-9d1c-9197dc8ad6bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:43.986860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-239cca9f-6b85-4f25-9d1c-9197dc8ad6bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:43.987289 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-239cca9f-6b85-4f25-9d1c-9197dc8ad6bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:44.045170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-239cca9f-6b85-4f25-9d1c-9197dc8ad6bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:44.054137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-239cca9f-6b85-4f25-9d1c-9197dc8ad6bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:44.055626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-239cca9f-6b85-4f25-9d1c-9197dc8ad6bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:44.056759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-239cca9f-6b85-4f25-9d1c-9197dc8ad6bc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082679 Jun 10 13:26:44.057342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1034/1948] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:43 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:45.074625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9c3f177c-7627-4ad1-bacd-a9b3e5995951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:45.075295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9c3f177c-7627-4ad1-bacd-a9b3e5995951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:45.082274 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c3f177c-7627-4ad1-bacd-a9b3e5995951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:45.082713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c3f177c-7627-4ad1-bacd-a9b3e5995951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:45.083486 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c3f177c-7627-4ad1-bacd-a9b3e5995951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:45.083913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c3f177c-7627-4ad1-bacd-a9b3e5995951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:45.144861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9c3f177c-7627-4ad1-bacd-a9b3e5995951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:45.153727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9c3f177c-7627-4ad1-bacd-a9b3e5995951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:45.155229 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9c3f177c-7627-4ad1-bacd-a9b3e5995951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:45.156397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9c3f177c-7627-4ad1-bacd-a9b3e5995951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085640 Jun 10 13:26:45.157008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1035/1949] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:45 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:46.174802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-94058dd2-ea82-46ff-a892-7c69200910d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:46.175496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-94058dd2-ea82-46ff-a892-7c69200910d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:46.181899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-94058dd2-ea82-46ff-a892-7c69200910d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:46.182362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-94058dd2-ea82-46ff-a892-7c69200910d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:46.183114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-94058dd2-ea82-46ff-a892-7c69200910d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:46.183549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-94058dd2-ea82-46ff-a892-7c69200910d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:46.244251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-94058dd2-ea82-46ff-a892-7c69200910d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:46.253245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-94058dd2-ea82-46ff-a892-7c69200910d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:46.254744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-94058dd2-ea82-46ff-a892-7c69200910d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:46.255881 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-94058dd2-ea82-46ff-a892-7c69200910d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085536 Jun 10 13:26:46.256439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1036/1950] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:46 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:47.273130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-557fbe66-c2d0-4b3e-84f0-19235fa1de8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:47.273889 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-557fbe66-c2d0-4b3e-84f0-19235fa1de8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:47.280347 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-557fbe66-c2d0-4b3e-84f0-19235fa1de8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:47.284675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-557fbe66-c2d0-4b3e-84f0-19235fa1de8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:47.284675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-557fbe66-c2d0-4b3e-84f0-19235fa1de8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:47.284675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-557fbe66-c2d0-4b3e-84f0-19235fa1de8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:47.343395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-557fbe66-c2d0-4b3e-84f0-19235fa1de8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:47.351696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-557fbe66-c2d0-4b3e-84f0-19235fa1de8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:47.353198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-557fbe66-c2d0-4b3e-84f0-19235fa1de8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:47.354335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-557fbe66-c2d0-4b3e-84f0-19235fa1de8c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085071 Jun 10 13:26:47.354879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1037/1951] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:47 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:48.373078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-57018969-3070-4e54-bf12-facc4cd4f2d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:48.373686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-57018969-3070-4e54-bf12-facc4cd4f2d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:48.380866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-57018969-3070-4e54-bf12-facc4cd4f2d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:48.381311 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-57018969-3070-4e54-bf12-facc4cd4f2d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:48.382058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-57018969-3070-4e54-bf12-facc4cd4f2d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:48.382511 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-57018969-3070-4e54-bf12-facc4cd4f2d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:48.441550 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-57018969-3070-4e54-bf12-facc4cd4f2d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:48.449737 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-57018969-3070-4e54-bf12-facc4cd4f2d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:48.451214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-57018969-3070-4e54-bf12-facc4cd4f2d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:48.452335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-57018969-3070-4e54-bf12-facc4cd4f2d3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083236 Jun 10 13:26:48.452911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1038/1952] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:48 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:49.471244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2d842955-3050-4ef9-8a39-93de1c914931 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:49.471877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2d842955-3050-4ef9-8a39-93de1c914931 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:49.478363 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d842955-3050-4ef9-8a39-93de1c914931 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:49.478809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d842955-3050-4ef9-8a39-93de1c914931 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:49.479608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d842955-3050-4ef9-8a39-93de1c914931 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:49.480036 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d842955-3050-4ef9-8a39-93de1c914931 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:49.542194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2d842955-3050-4ef9-8a39-93de1c914931 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:49.550655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2d842955-3050-4ef9-8a39-93de1c914931 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:49.552487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2d842955-3050-4ef9-8a39-93de1c914931 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:49.553645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2d842955-3050-4ef9-8a39-93de1c914931 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087426 Jun 10 13:26:49.554190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1039/1953] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:49 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:50.571914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-71839824-2e1e-405b-ba46-bd7c8c531f1f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:50.572531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-71839824-2e1e-405b-ba46-bd7c8c531f1f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:50.578992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71839824-2e1e-405b-ba46-bd7c8c531f1f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:50.579432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71839824-2e1e-405b-ba46-bd7c8c531f1f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:50.580180 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71839824-2e1e-405b-ba46-bd7c8c531f1f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:50.580606 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71839824-2e1e-405b-ba46-bd7c8c531f1f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:50.639450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-71839824-2e1e-405b-ba46-bd7c8c531f1f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:50.647543 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-71839824-2e1e-405b-ba46-bd7c8c531f1f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:50.649058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-71839824-2e1e-405b-ba46-bd7c8c531f1f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:50.650179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-71839824-2e1e-405b-ba46-bd7c8c531f1f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082160 Jun 10 13:26:50.650731 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1040/1954] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:50 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:51.669149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-71f61ade-e1ec-4270-852e-864f14871ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:51.669149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-71f61ade-e1ec-4270-852e-864f14871ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:51.676216 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71f61ade-e1ec-4270-852e-864f14871ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:51.676661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71f61ade-e1ec-4270-852e-864f14871ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:51.677478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71f61ade-e1ec-4270-852e-864f14871ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:51.677905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71f61ade-e1ec-4270-852e-864f14871ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:51.737578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-71f61ade-e1ec-4270-852e-864f14871ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:51.746094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-71f61ade-e1ec-4270-852e-864f14871ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:51.747583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-71f61ade-e1ec-4270-852e-864f14871ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:51.748788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-71f61ade-e1ec-4270-852e-864f14871ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084390 Jun 10 13:26:51.749335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1041/1955] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:51 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:52.766027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cbf01235-570c-4d25-96c9-994505af78a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:52.766639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cbf01235-570c-4d25-96c9-994505af78a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:52.773115 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cbf01235-570c-4d25-96c9-994505af78a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:52.773570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cbf01235-570c-4d25-96c9-994505af78a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:52.774321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cbf01235-570c-4d25-96c9-994505af78a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:52.774752 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cbf01235-570c-4d25-96c9-994505af78a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:52.835318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cbf01235-570c-4d25-96c9-994505af78a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:52.844180 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cbf01235-570c-4d25-96c9-994505af78a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:52.845693 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cbf01235-570c-4d25-96c9-994505af78a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:52.846834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cbf01235-570c-4d25-96c9-994505af78a6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085284 Jun 10 13:26:52.847393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1042/1956] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:52 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:53.864597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d908df0d-115f-46c2-993e-81a098d46dfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:53.865261 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d908df0d-115f-46c2-993e-81a098d46dfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:53.871609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d908df0d-115f-46c2-993e-81a098d46dfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:53.872052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d908df0d-115f-46c2-993e-81a098d46dfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:53.872826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d908df0d-115f-46c2-993e-81a098d46dfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:53.873258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d908df0d-115f-46c2-993e-81a098d46dfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:53.934226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d908df0d-115f-46c2-993e-81a098d46dfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:53.942481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d908df0d-115f-46c2-993e-81a098d46dfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:53.943952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d908df0d-115f-46c2-993e-81a098d46dfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:53.945148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d908df0d-115f-46c2-993e-81a098d46dfc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084389 Jun 10 13:26:53.945692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1043/1957] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:53 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:54.204222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:26:54.204436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:26:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:26:54.963446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b5ba9627-da7c-4ae5-bcf2-53d07adabfe0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:54.964096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b5ba9627-da7c-4ae5-bcf2-53d07adabfe0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:54.971436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5ba9627-da7c-4ae5-bcf2-53d07adabfe0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:54.971875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5ba9627-da7c-4ae5-bcf2-53d07adabfe0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:54.972622 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5ba9627-da7c-4ae5-bcf2-53d07adabfe0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:54.973077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5ba9627-da7c-4ae5-bcf2-53d07adabfe0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:55.034127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b5ba9627-da7c-4ae5-bcf2-53d07adabfe0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:55.042629 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b5ba9627-da7c-4ae5-bcf2-53d07adabfe0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:55.044561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b5ba9627-da7c-4ae5-bcf2-53d07adabfe0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:55.045298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b5ba9627-da7c-4ae5-bcf2-53d07adabfe0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085787 Jun 10 13:26:55.045843 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1044/1958] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:54 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:56.065920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cf263145-fc07-486f-8a02-068868d68b25 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:56.066602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cf263145-fc07-486f-8a02-068868d68b25 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:56.072958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf263145-fc07-486f-8a02-068868d68b25 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:56.073404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf263145-fc07-486f-8a02-068868d68b25 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:56.074163 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf263145-fc07-486f-8a02-068868d68b25 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:56.074592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cf263145-fc07-486f-8a02-068868d68b25 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:56.133926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cf263145-fc07-486f-8a02-068868d68b25 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:56.142624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cf263145-fc07-486f-8a02-068868d68b25 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:56.144126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cf263145-fc07-486f-8a02-068868d68b25 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:56.145292 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cf263145-fc07-486f-8a02-068868d68b25 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083307 Jun 10 13:26:56.145840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1045/1959] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:56 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:57.163463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-834d6b46-6e07-4538-ac5c-6b91a674874e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:57.164095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-834d6b46-6e07-4538-ac5c-6b91a674874e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:57.170479 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-834d6b46-6e07-4538-ac5c-6b91a674874e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:57.170931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-834d6b46-6e07-4538-ac5c-6b91a674874e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:57.172334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-834d6b46-6e07-4538-ac5c-6b91a674874e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:57.172794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-834d6b46-6e07-4538-ac5c-6b91a674874e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:57.232099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-834d6b46-6e07-4538-ac5c-6b91a674874e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:57.241077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-834d6b46-6e07-4538-ac5c-6b91a674874e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:57.242568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-834d6b46-6e07-4538-ac5c-6b91a674874e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:57.243703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-834d6b46-6e07-4538-ac5c-6b91a674874e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084229 Jun 10 13:26:57.244249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1046/1960] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:57 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:58.261491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dc522970-5304-4623-b874-dc314bdb4672 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:58.262118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dc522970-5304-4623-b874-dc314bdb4672 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:58.269214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc522970-5304-4623-b874-dc314bdb4672 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:58.269658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc522970-5304-4623-b874-dc314bdb4672 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:58.270447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc522970-5304-4623-b874-dc314bdb4672 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:58.270987 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc522970-5304-4623-b874-dc314bdb4672 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:58.335550 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dc522970-5304-4623-b874-dc314bdb4672 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:58.344304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dc522970-5304-4623-b874-dc314bdb4672 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:58.345833 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dc522970-5304-4623-b874-dc314bdb4672 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:58.346969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dc522970-5304-4623-b874-dc314bdb4672 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.089509 Jun 10 13:26:58.347525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1047/1961] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:58 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:26:59.364744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-83a54b8c-beab-42fc-aec3-aa2aee2c771b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:26:59.365367 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-83a54b8c-beab-42fc-aec3-aa2aee2c771b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:26:59.371792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-83a54b8c-beab-42fc-aec3-aa2aee2c771b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:59.372236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-83a54b8c-beab-42fc-aec3-aa2aee2c771b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:59.373011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-83a54b8c-beab-42fc-aec3-aa2aee2c771b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:26:59.373437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-83a54b8c-beab-42fc-aec3-aa2aee2c771b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:26:59.377809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:26:59.436068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-83a54b8c-beab-42fc-aec3-aa2aee2c771b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:59.444979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-83a54b8c-beab-42fc-aec3-aa2aee2c771b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:59.446466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-83a54b8c-beab-42fc-aec3-aa2aee2c771b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:26:59.447654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-83a54b8c-beab-42fc-aec3-aa2aee2c771b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086799 Jun 10 13:26:59.448282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1048/1962] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:26:59 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:00.466815 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f1b679aa-896a-4b5d-bd0a-5a8feac7ae12 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:00.467421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f1b679aa-896a-4b5d-bd0a-5a8feac7ae12 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:00.474492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1b679aa-896a-4b5d-bd0a-5a8feac7ae12 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:00.474932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1b679aa-896a-4b5d-bd0a-5a8feac7ae12 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:00.475676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1b679aa-896a-4b5d-bd0a-5a8feac7ae12 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:00.476105 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1b679aa-896a-4b5d-bd0a-5a8feac7ae12 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:00.536349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f1b679aa-896a-4b5d-bd0a-5a8feac7ae12 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:00.544997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f1b679aa-896a-4b5d-bd0a-5a8feac7ae12 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:00.546517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f1b679aa-896a-4b5d-bd0a-5a8feac7ae12 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:00.547634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f1b679aa-896a-4b5d-bd0a-5a8feac7ae12 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085032 Jun 10 13:27:00.548181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1049/1963] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:00 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:01.565393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f1c64bce-328d-4670-9f7a-1c650fc3a2dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:01.566065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f1c64bce-328d-4670-9f7a-1c650fc3a2dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:01.571988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1c64bce-328d-4670-9f7a-1c650fc3a2dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:01.572439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1c64bce-328d-4670-9f7a-1c650fc3a2dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:01.573231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1c64bce-328d-4670-9f7a-1c650fc3a2dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:01.573673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1c64bce-328d-4670-9f7a-1c650fc3a2dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:01.633394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f1c64bce-328d-4670-9f7a-1c650fc3a2dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:01.641686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f1c64bce-328d-4670-9f7a-1c650fc3a2dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:01.643188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f1c64bce-328d-4670-9f7a-1c650fc3a2dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:01.644326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f1c64bce-328d-4670-9f7a-1c650fc3a2dc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082945 Jun 10 13:27:01.644911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1050/1964] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:01 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:02.662791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e19ba3ab-bcb0-4353-af6e-e77187e4a02f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:02.663522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e19ba3ab-bcb0-4353-af6e-e77187e4a02f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:02.670376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e19ba3ab-bcb0-4353-af6e-e77187e4a02f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:02.670819 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e19ba3ab-bcb0-4353-af6e-e77187e4a02f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:02.672081 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e19ba3ab-bcb0-4353-af6e-e77187e4a02f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:02.672512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e19ba3ab-bcb0-4353-af6e-e77187e4a02f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:02.738747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e19ba3ab-bcb0-4353-af6e-e77187e4a02f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:02.750919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e19ba3ab-bcb0-4353-af6e-e77187e4a02f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:02.752424 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e19ba3ab-bcb0-4353-af6e-e77187e4a02f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:02.753656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e19ba3ab-bcb0-4353-af6e-e77187e4a02f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.095255 Jun 10 13:27:02.754215 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1051/1965] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:02 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:03.772074 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e716bc15-e461-47ac-83f7-9f15e697bc96 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:03.772717 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e716bc15-e461-47ac-83f7-9f15e697bc96 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:03.779390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e716bc15-e461-47ac-83f7-9f15e697bc96 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:03.779837 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e716bc15-e461-47ac-83f7-9f15e697bc96 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:03.780599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e716bc15-e461-47ac-83f7-9f15e697bc96 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:03.781068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e716bc15-e461-47ac-83f7-9f15e697bc96 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:03.840097 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e716bc15-e461-47ac-83f7-9f15e697bc96 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:03.848726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e716bc15-e461-47ac-83f7-9f15e697bc96 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:03.850250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e716bc15-e461-47ac-83f7-9f15e697bc96 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:03.851379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e716bc15-e461-47ac-83f7-9f15e697bc96 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083229 Jun 10 13:27:03.851932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1052/1966] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:03 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:04.869362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1de0e34f-99ee-416e-8645-c8d7ffaa39aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:04.870034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1de0e34f-99ee-416e-8645-c8d7ffaa39aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:04.876741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1de0e34f-99ee-416e-8645-c8d7ffaa39aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:04.877208 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1de0e34f-99ee-416e-8645-c8d7ffaa39aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:04.877977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1de0e34f-99ee-416e-8645-c8d7ffaa39aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:04.878409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1de0e34f-99ee-416e-8645-c8d7ffaa39aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:04.937268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1de0e34f-99ee-416e-8645-c8d7ffaa39aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:04.946199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1de0e34f-99ee-416e-8645-c8d7ffaa39aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:04.947698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1de0e34f-99ee-416e-8645-c8d7ffaa39aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:04.948873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1de0e34f-99ee-416e-8645-c8d7ffaa39aa tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083430 Jun 10 13:27:04.949431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1053/1967] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:04 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:05.965997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-79358549-7ad1-489e-a984-e7773505a864 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:05.966934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-79358549-7ad1-489e-a984-e7773505a864 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:05.973969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79358549-7ad1-489e-a984-e7773505a864 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:05.974411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79358549-7ad1-489e-a984-e7773505a864 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:05.975160 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79358549-7ad1-489e-a984-e7773505a864 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:05.975589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79358549-7ad1-489e-a984-e7773505a864 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:06.034311 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-79358549-7ad1-489e-a984-e7773505a864 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:06.042402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-79358549-7ad1-489e-a984-e7773505a864 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:06.043885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-79358549-7ad1-489e-a984-e7773505a864 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:06.045113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-79358549-7ad1-489e-a984-e7773505a864 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082966 Jun 10 13:27:06.045670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1054/1968] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:05 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:07.062985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-53f32e27-badd-4d89-bc44-c5465efb90b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:07.063651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-53f32e27-badd-4d89-bc44-c5465efb90b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:07.065914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=56 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:27:07.070625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53f32e27-badd-4d89-bc44-c5465efb90b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:07.071078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53f32e27-badd-4d89-bc44-c5465efb90b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:07.071837 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53f32e27-badd-4d89-bc44-c5465efb90b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:07.072270 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53f32e27-badd-4d89-bc44-c5465efb90b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:07.134275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-53f32e27-badd-4d89-bc44-c5465efb90b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:07.143059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-53f32e27-badd-4d89-bc44-c5465efb90b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:07.144553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-53f32e27-badd-4d89-bc44-c5465efb90b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:07.145724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-53f32e27-badd-4d89-bc44-c5465efb90b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086698 Jun 10 13:27:07.146287 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1055/1969] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:07 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:08.163500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6377628e-dc71-43d0-a2cd-76dfc3747951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:08.164187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6377628e-dc71-43d0-a2cd-76dfc3747951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:08.170821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6377628e-dc71-43d0-a2cd-76dfc3747951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:08.171272 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6377628e-dc71-43d0-a2cd-76dfc3747951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:08.172596 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6377628e-dc71-43d0-a2cd-76dfc3747951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:08.173054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6377628e-dc71-43d0-a2cd-76dfc3747951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:08.233639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6377628e-dc71-43d0-a2cd-76dfc3747951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:08.242465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6377628e-dc71-43d0-a2cd-76dfc3747951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:08.243957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6377628e-dc71-43d0-a2cd-76dfc3747951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:08.245120 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6377628e-dc71-43d0-a2cd-76dfc3747951 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085517 Jun 10 13:27:08.245682 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1056/1970] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:08 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:09.263370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5e08cdc2-5887-4192-bc7c-b243da436798 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:09.264061 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5e08cdc2-5887-4192-bc7c-b243da436798 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:09.270566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e08cdc2-5887-4192-bc7c-b243da436798 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:09.271008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e08cdc2-5887-4192-bc7c-b243da436798 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:09.271749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e08cdc2-5887-4192-bc7c-b243da436798 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:09.272180 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e08cdc2-5887-4192-bc7c-b243da436798 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:09.331308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5e08cdc2-5887-4192-bc7c-b243da436798 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:09.339869 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5e08cdc2-5887-4192-bc7c-b243da436798 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:09.341743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5e08cdc2-5887-4192-bc7c-b243da436798 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:09.342882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5e08cdc2-5887-4192-bc7c-b243da436798 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083417 Jun 10 13:27:09.343445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1057/1971] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:09 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:10.361114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2ec84f25-7954-4732-8516-dfdac9fa4ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:10.361734 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2ec84f25-7954-4732-8516-dfdac9fa4ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:10.372960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ec84f25-7954-4732-8516-dfdac9fa4ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:10.373415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ec84f25-7954-4732-8516-dfdac9fa4ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:10.374206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ec84f25-7954-4732-8516-dfdac9fa4ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:10.374643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ec84f25-7954-4732-8516-dfdac9fa4ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:10.438087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2ec84f25-7954-4732-8516-dfdac9fa4ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:10.446350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2ec84f25-7954-4732-8516-dfdac9fa4ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:10.447851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2ec84f25-7954-4732-8516-dfdac9fa4ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:10.449019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2ec84f25-7954-4732-8516-dfdac9fa4ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.092285 Jun 10 13:27:10.449574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1058/1972] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:10 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:11.467007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6dec5984-030e-4c16-bc9a-03130c4b32a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:11.467777 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6dec5984-030e-4c16-bc9a-03130c4b32a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:11.474152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6dec5984-030e-4c16-bc9a-03130c4b32a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:11.474606 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6dec5984-030e-4c16-bc9a-03130c4b32a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:11.475373 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6dec5984-030e-4c16-bc9a-03130c4b32a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:11.475808 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6dec5984-030e-4c16-bc9a-03130c4b32a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:11.537365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6dec5984-030e-4c16-bc9a-03130c4b32a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:11.545630 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6dec5984-030e-4c16-bc9a-03130c4b32a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:11.547165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6dec5984-030e-4c16-bc9a-03130c4b32a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:11.548326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6dec5984-030e-4c16-bc9a-03130c4b32a7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085506 Jun 10 13:27:11.548911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1059/1973] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:11 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:12.568195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-063b025d-c8f4-415e-9ca0-3d44f46149ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:12.568823 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-063b025d-c8f4-415e-9ca0-3d44f46149ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:12.575349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-063b025d-c8f4-415e-9ca0-3d44f46149ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:12.575799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-063b025d-c8f4-415e-9ca0-3d44f46149ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:12.576553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-063b025d-c8f4-415e-9ca0-3d44f46149ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:12.577012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-063b025d-c8f4-415e-9ca0-3d44f46149ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:12.635650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-063b025d-c8f4-415e-9ca0-3d44f46149ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:12.644213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-063b025d-c8f4-415e-9ca0-3d44f46149ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:12.645726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-063b025d-c8f4-415e-9ca0-3d44f46149ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:12.647066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-063b025d-c8f4-415e-9ca0-3d44f46149ee tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082599 Jun 10 13:27:12.648021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1060/1974] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:12 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:13.666359 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-175d6921-4363-4bea-8b19-1d6a337b8098 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:13.667014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-175d6921-4363-4bea-8b19-1d6a337b8098 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:13.674001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-175d6921-4363-4bea-8b19-1d6a337b8098 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:13.674440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-175d6921-4363-4bea-8b19-1d6a337b8098 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:13.675191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-175d6921-4363-4bea-8b19-1d6a337b8098 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:13.675619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-175d6921-4363-4bea-8b19-1d6a337b8098 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:13.733611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-175d6921-4363-4bea-8b19-1d6a337b8098 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:13.742517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-175d6921-4363-4bea-8b19-1d6a337b8098 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:13.743995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-175d6921-4363-4bea-8b19-1d6a337b8098 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:13.745145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-175d6921-4363-4bea-8b19-1d6a337b8098 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082752 Jun 10 13:27:13.745702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1061/1975] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:13 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:14.763406 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2eaec292-7846-483f-8ae0-548702dd6f39 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:14.763997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2eaec292-7846-483f-8ae0-548702dd6f39 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:14.770948 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2eaec292-7846-483f-8ae0-548702dd6f39 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:14.771395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2eaec292-7846-483f-8ae0-548702dd6f39 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:14.772147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2eaec292-7846-483f-8ae0-548702dd6f39 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:14.772582 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2eaec292-7846-483f-8ae0-548702dd6f39 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:14.833076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2eaec292-7846-483f-8ae0-548702dd6f39 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:14.841764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2eaec292-7846-483f-8ae0-548702dd6f39 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:14.843251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2eaec292-7846-483f-8ae0-548702dd6f39 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:14.844386 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2eaec292-7846-483f-8ae0-548702dd6f39 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084972 Jun 10 13:27:14.845530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1062/1976] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:14 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:15.863767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e4dbf0eb-2127-43b0-bc17-d7803a1dbee7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:15.864431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e4dbf0eb-2127-43b0-bc17-d7803a1dbee7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:15.870923 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4dbf0eb-2127-43b0-bc17-d7803a1dbee7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:15.871366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4dbf0eb-2127-43b0-bc17-d7803a1dbee7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:15.872107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4dbf0eb-2127-43b0-bc17-d7803a1dbee7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:15.872533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4dbf0eb-2127-43b0-bc17-d7803a1dbee7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:15.933060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e4dbf0eb-2127-43b0-bc17-d7803a1dbee7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:15.941363 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e4dbf0eb-2127-43b0-bc17-d7803a1dbee7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:15.942848 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e4dbf0eb-2127-43b0-bc17-d7803a1dbee7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:15.943967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e4dbf0eb-2127-43b0-bc17-d7803a1dbee7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084641 Jun 10 13:27:15.944511 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1063/1977] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:15 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:16.961593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2d2bd9c6-f668-4d09-9607-caa14e57ab8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:16.962254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2d2bd9c6-f668-4d09-9607-caa14e57ab8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:16.968962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d2bd9c6-f668-4d09-9607-caa14e57ab8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:16.969415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d2bd9c6-f668-4d09-9607-caa14e57ab8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:16.970209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d2bd9c6-f668-4d09-9607-caa14e57ab8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:16.970636 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d2bd9c6-f668-4d09-9607-caa14e57ab8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:17.029556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2d2bd9c6-f668-4d09-9607-caa14e57ab8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:17.032952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:27:17.038091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2d2bd9c6-f668-4d09-9607-caa14e57ab8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:17.039594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2d2bd9c6-f668-4d09-9607-caa14e57ab8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:17.040726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2d2bd9c6-f668-4d09-9607-caa14e57ab8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083086 Jun 10 13:27:17.043018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1064/1978] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:16 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:18.059489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-26c6452e-08ca-46a7-84f4-0c2a3d05399f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:18.060179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-26c6452e-08ca-46a7-84f4-0c2a3d05399f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:18.066695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26c6452e-08ca-46a7-84f4-0c2a3d05399f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:18.067141 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26c6452e-08ca-46a7-84f4-0c2a3d05399f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:18.067884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26c6452e-08ca-46a7-84f4-0c2a3d05399f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:18.068313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26c6452e-08ca-46a7-84f4-0c2a3d05399f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:18.126977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-26c6452e-08ca-46a7-84f4-0c2a3d05399f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:18.135166 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-26c6452e-08ca-46a7-84f4-0c2a3d05399f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:18.136686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-26c6452e-08ca-46a7-84f4-0c2a3d05399f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:18.137840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-26c6452e-08ca-46a7-84f4-0c2a3d05399f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082607 Jun 10 13:27:18.138398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1065/1979] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:18 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:19.155617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4bbcf9ac-9cc8-44b1-9aad-011a3b77dfca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:19.156266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4bbcf9ac-9cc8-44b1-9aad-011a3b77dfca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:19.162658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4bbcf9ac-9cc8-44b1-9aad-011a3b77dfca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:19.163135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4bbcf9ac-9cc8-44b1-9aad-011a3b77dfca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:19.163892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4bbcf9ac-9cc8-44b1-9aad-011a3b77dfca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:19.164329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4bbcf9ac-9cc8-44b1-9aad-011a3b77dfca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:19.224286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4bbcf9ac-9cc8-44b1-9aad-011a3b77dfca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:19.232787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4bbcf9ac-9cc8-44b1-9aad-011a3b77dfca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:19.234333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4bbcf9ac-9cc8-44b1-9aad-011a3b77dfca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:19.236061 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4bbcf9ac-9cc8-44b1-9aad-011a3b77dfca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083992 Jun 10 13:27:19.236958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1066/1980] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:19 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:20.254206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0af90563-a4ba-46d8-b898-9550771c24d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:20.254869 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0af90563-a4ba-46d8-b898-9550771c24d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:20.261320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0af90563-a4ba-46d8-b898-9550771c24d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:20.261760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0af90563-a4ba-46d8-b898-9550771c24d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:20.262504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0af90563-a4ba-46d8-b898-9550771c24d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:20.262928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0af90563-a4ba-46d8-b898-9550771c24d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:20.324704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0af90563-a4ba-46d8-b898-9550771c24d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:20.333160 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0af90563-a4ba-46d8-b898-9550771c24d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:20.334670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0af90563-a4ba-46d8-b898-9550771c24d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:20.335784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0af90563-a4ba-46d8-b898-9550771c24d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085995 Jun 10 13:27:20.336856 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1067/1981] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:20 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:21.354676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-81d650cf-387a-4d50-85ab-5380ea29f574 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:21.355288 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-81d650cf-387a-4d50-85ab-5380ea29f574 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:21.361968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-81d650cf-387a-4d50-85ab-5380ea29f574 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:21.362422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-81d650cf-387a-4d50-85ab-5380ea29f574 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:21.363222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-81d650cf-387a-4d50-85ab-5380ea29f574 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:21.363668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-81d650cf-387a-4d50-85ab-5380ea29f574 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:21.424857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-81d650cf-387a-4d50-85ab-5380ea29f574 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:21.433265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-81d650cf-387a-4d50-85ab-5380ea29f574 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:21.434754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-81d650cf-387a-4d50-85ab-5380ea29f574 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:21.435872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-81d650cf-387a-4d50-85ab-5380ea29f574 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085037 Jun 10 13:27:21.436422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1068/1982] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:21 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:21.440715 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-922f2a62-6fa2-467c-9134-b841fce21b67 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:21.441224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-922f2a62-6fa2-467c-9134-b841fce21b67 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:21.446959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-922f2a62-6fa2-467c-9134-b841fce21b67 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:21.447378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-922f2a62-6fa2-467c-9134-b841fce21b67 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:21.448130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-922f2a62-6fa2-467c-9134-b841fce21b67 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:21.448557 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-922f2a62-6fa2-467c-9134-b841fce21b67 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:21.475705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-922f2a62-6fa2-467c-9134-b841fce21b67 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:27:21.738448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-922f2a62-6fa2-467c-9134-b841fce21b67 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 204 len: 0 microversion: 2.1 time: 0.300752 Jun 10 13:27:21.739038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1069/1983] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:27:21 2022] DELETE /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jun 10 13:27:21.747312 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a93c4daa-b573-4d24-b1c5-af5b0fd4743d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:21.748006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a93c4daa-b573-4d24-b1c5-af5b0fd4743d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:21.754126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a93c4daa-b573-4d24-b1c5-af5b0fd4743d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:21.754572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a93c4daa-b573-4d24-b1c5-af5b0fd4743d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:21.755323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a93c4daa-b573-4d24-b1c5-af5b0fd4743d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:21.755755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a93c4daa-b573-4d24-b1c5-af5b0fd4743d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:21.881140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a93c4daa-b573-4d24-b1c5-af5b0fd4743d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:21.889140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a93c4daa-b573-4d24-b1c5-af5b0fd4743d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145747 Jun 10 13:27:21.889704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1070/1984] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:21 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:22.455993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2f9b8a37-7d35-4ce7-a012-35664e2a6356 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:22.456595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2f9b8a37-7d35-4ce7-a012-35664e2a6356 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:22.463090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f9b8a37-7d35-4ce7-a012-35664e2a6356 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:22.463525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f9b8a37-7d35-4ce7-a012-35664e2a6356 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:22.464264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f9b8a37-7d35-4ce7-a012-35664e2a6356 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:22.464691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f9b8a37-7d35-4ce7-a012-35664e2a6356 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:22.526919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2f9b8a37-7d35-4ce7-a012-35664e2a6356 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:22.535552 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2f9b8a37-7d35-4ce7-a012-35664e2a6356 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:22.537128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2f9b8a37-7d35-4ce7-a012-35664e2a6356 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:22.538264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2f9b8a37-7d35-4ce7-a012-35664e2a6356 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.088247 Jun 10 13:27:22.538831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1071/1985] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:22 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:22.905792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-70f09edb-f10f-442d-8647-adea84c37b72 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:22.906396 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-70f09edb-f10f-442d-8647-adea84c37b72 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:22.912539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-70f09edb-f10f-442d-8647-adea84c37b72 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:22.913017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-70f09edb-f10f-442d-8647-adea84c37b72 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:22.913767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-70f09edb-f10f-442d-8647-adea84c37b72 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:22.914192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-70f09edb-f10f-442d-8647-adea84c37b72 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:23.041946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-70f09edb-f10f-442d-8647-adea84c37b72 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:23.044339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-70f09edb-f10f-442d-8647-adea84c37b72 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.142308 Jun 10 13:27:23.045012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1072/1986] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:22 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:27:23.555927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-de2bdec3-28ce-46f2-8c88-125b01a7bbca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:23.556527 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-de2bdec3-28ce-46f2-8c88-125b01a7bbca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:23.562930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-de2bdec3-28ce-46f2-8c88-125b01a7bbca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:23.563934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-de2bdec3-28ce-46f2-8c88-125b01a7bbca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:23.564680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-de2bdec3-28ce-46f2-8c88-125b01a7bbca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:23.565132 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-de2bdec3-28ce-46f2-8c88-125b01a7bbca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:23.624788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-de2bdec3-28ce-46f2-8c88-125b01a7bbca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:23.633532 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-de2bdec3-28ce-46f2-8c88-125b01a7bbca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:23.635006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-de2bdec3-28ce-46f2-8c88-125b01a7bbca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:23.636127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-de2bdec3-28ce-46f2-8c88-125b01a7bbca tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084078 Jun 10 13:27:23.636681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1073/1987] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:23 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:24.062031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-232d12b8-67e9-4118-9bc7-13c713471c74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:24.062633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-232d12b8-67e9-4118-9bc7-13c713471c74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:24.068603 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-232d12b8-67e9-4118-9bc7-13c713471c74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:24.069077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-232d12b8-67e9-4118-9bc7-13c713471c74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:24.069822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-232d12b8-67e9-4118-9bc7-13c713471c74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:24.070249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-232d12b8-67e9-4118-9bc7-13c713471c74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:24.199589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-232d12b8-67e9-4118-9bc7-13c713471c74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:24.207618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-232d12b8-67e9-4118-9bc7-13c713471c74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149481 Jun 10 13:27:24.208217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1074/1988] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:24 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:24.652958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7fff11c1-2e71-4eaa-8d0d-f7d402702c8f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:24.653589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7fff11c1-2e71-4eaa-8d0d-f7d402702c8f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:24.660077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7fff11c1-2e71-4eaa-8d0d-f7d402702c8f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:24.660525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7fff11c1-2e71-4eaa-8d0d-f7d402702c8f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:24.661323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7fff11c1-2e71-4eaa-8d0d-f7d402702c8f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:24.661763 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7fff11c1-2e71-4eaa-8d0d-f7d402702c8f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:24.722196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7fff11c1-2e71-4eaa-8d0d-f7d402702c8f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:24.730544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7fff11c1-2e71-4eaa-8d0d-f7d402702c8f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:24.732035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7fff11c1-2e71-4eaa-8d0d-f7d402702c8f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:24.733231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7fff11c1-2e71-4eaa-8d0d-f7d402702c8f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084229 Jun 10 13:27:24.733790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1075/1989] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:24 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:25.234761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5f226e9b-124d-490b-82bb-f2a76099f5ac tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:25.235418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5f226e9b-124d-490b-82bb-f2a76099f5ac tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:25.241449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5f226e9b-124d-490b-82bb-f2a76099f5ac tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:25.241892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5f226e9b-124d-490b-82bb-f2a76099f5ac tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:25.242635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5f226e9b-124d-490b-82bb-f2a76099f5ac tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:25.243063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5f226e9b-124d-490b-82bb-f2a76099f5ac tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:25.374203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5f226e9b-124d-490b-82bb-f2a76099f5ac tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:25.382306 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5f226e9b-124d-490b-82bb-f2a76099f5ac tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.151368 Jun 10 13:27:25.382867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1076/1990] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:25 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:25.751919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bd9035e7-b10f-4f0f-8b80-545a9c183b79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:25.752574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bd9035e7-b10f-4f0f-8b80-545a9c183b79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:25.758995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bd9035e7-b10f-4f0f-8b80-545a9c183b79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:25.759443 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bd9035e7-b10f-4f0f-8b80-545a9c183b79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:25.760197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bd9035e7-b10f-4f0f-8b80-545a9c183b79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:25.760627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bd9035e7-b10f-4f0f-8b80-545a9c183b79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:25.822319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bd9035e7-b10f-4f0f-8b80-545a9c183b79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:25.831207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bd9035e7-b10f-4f0f-8b80-545a9c183b79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:25.832700 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bd9035e7-b10f-4f0f-8b80-545a9c183b79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:25.833865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bd9035e7-b10f-4f0f-8b80-545a9c183b79 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086375 Jun 10 13:27:25.834418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1077/1991] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:25 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:26.399623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-21cd7d66-d0bb-4cb6-85dc-f0eb87657002 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:26.400229 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-21cd7d66-d0bb-4cb6-85dc-f0eb87657002 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:26.406251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-21cd7d66-d0bb-4cb6-85dc-f0eb87657002 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:26.406697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-21cd7d66-d0bb-4cb6-85dc-f0eb87657002 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:26.407451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-21cd7d66-d0bb-4cb6-85dc-f0eb87657002 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:26.407881 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-21cd7d66-d0bb-4cb6-85dc-f0eb87657002 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:26.530291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-21cd7d66-d0bb-4cb6-85dc-f0eb87657002 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:26.538137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-21cd7d66-d0bb-4cb6-85dc-f0eb87657002 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.142408 Jun 10 13:27:26.538697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1078/1992] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:26 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:26.851830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6021c987-7b22-4305-ad00-ae4a251ab9c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:26.852499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6021c987-7b22-4305-ad00-ae4a251ab9c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:26.858977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6021c987-7b22-4305-ad00-ae4a251ab9c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:26.859425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6021c987-7b22-4305-ad00-ae4a251ab9c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:26.860713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6021c987-7b22-4305-ad00-ae4a251ab9c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:26.861172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6021c987-7b22-4305-ad00-ae4a251ab9c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:26.920623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6021c987-7b22-4305-ad00-ae4a251ab9c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:26.924612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:27:26.929825 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6021c987-7b22-4305-ad00-ae4a251ab9c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:26.931365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6021c987-7b22-4305-ad00-ae4a251ab9c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:26.932498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6021c987-7b22-4305-ad00-ae4a251ab9c4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084634 Jun 10 13:27:26.933088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1079/1993] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:26 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:27.555208 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1074e177-e57d-49a2-91e1-23583ce75a39 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:27.555843 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1074e177-e57d-49a2-91e1-23583ce75a39 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:27.562168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1074e177-e57d-49a2-91e1-23583ce75a39 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:27.562617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1074e177-e57d-49a2-91e1-23583ce75a39 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:27.563370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1074e177-e57d-49a2-91e1-23583ce75a39 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:27.563799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1074e177-e57d-49a2-91e1-23583ce75a39 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:27.704479 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1074e177-e57d-49a2-91e1-23583ce75a39 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:27.712355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1074e177-e57d-49a2-91e1-23583ce75a39 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.161312 Jun 10 13:27:27.712945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1080/1994] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:27 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:27.951054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-59dbe5de-2995-4051-b5c3-e9c1c22839e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:27.951742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-59dbe5de-2995-4051-b5c3-e9c1c22839e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:27.958160 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-59dbe5de-2995-4051-b5c3-e9c1c22839e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:27.958599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-59dbe5de-2995-4051-b5c3-e9c1c22839e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:27.959361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-59dbe5de-2995-4051-b5c3-e9c1c22839e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:27.959793 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-59dbe5de-2995-4051-b5c3-e9c1c22839e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:28.020659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-59dbe5de-2995-4051-b5c3-e9c1c22839e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:28.028949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-59dbe5de-2995-4051-b5c3-e9c1c22839e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:28.030448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-59dbe5de-2995-4051-b5c3-e9c1c22839e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:28.031774 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-59dbe5de-2995-4051-b5c3-e9c1c22839e9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084430 Jun 10 13:27:28.032331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1081/1995] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:27 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:28.730025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a9df605f-6cf6-4605-94cf-0caf4dd898cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:28.739585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a9df605f-6cf6-4605-94cf-0caf4dd898cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:28.739585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a9df605f-6cf6-4605-94cf-0caf4dd898cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:28.739585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a9df605f-6cf6-4605-94cf-0caf4dd898cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:28.739585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a9df605f-6cf6-4605-94cf-0caf4dd898cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:28.739955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a9df605f-6cf6-4605-94cf-0caf4dd898cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:28.860956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a9df605f-6cf6-4605-94cf-0caf4dd898cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:28.868752 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a9df605f-6cf6-4605-94cf-0caf4dd898cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.142518 Jun 10 13:27:28.869465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1082/1996] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:28 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:29.049563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-36544d35-0a5a-434f-9cc1-319e203e662c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:29.050178 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-36544d35-0a5a-434f-9cc1-319e203e662c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:29.056536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36544d35-0a5a-434f-9cc1-319e203e662c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:29.057019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36544d35-0a5a-434f-9cc1-319e203e662c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:29.057777 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36544d35-0a5a-434f-9cc1-319e203e662c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:29.058211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36544d35-0a5a-434f-9cc1-319e203e662c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:29.119953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-36544d35-0a5a-434f-9cc1-319e203e662c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:29.128411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-36544d35-0a5a-434f-9cc1-319e203e662c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:29.129958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-36544d35-0a5a-434f-9cc1-319e203e662c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:29.131094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-36544d35-0a5a-434f-9cc1-319e203e662c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085408 Jun 10 13:27:29.131685 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1083/1997] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:29 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:29.884870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4bd5320d-5632-4536-907c-54b93c6e6435 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:29.885482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4bd5320d-5632-4536-907c-54b93c6e6435 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:29.892092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4bd5320d-5632-4536-907c-54b93c6e6435 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:29.892536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4bd5320d-5632-4536-907c-54b93c6e6435 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:29.893309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4bd5320d-5632-4536-907c-54b93c6e6435 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:29.893844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4bd5320d-5632-4536-907c-54b93c6e6435 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:30.018880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4bd5320d-5632-4536-907c-54b93c6e6435 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:30.033065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4bd5320d-5632-4536-907c-54b93c6e6435 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145846 Jun 10 13:27:30.033065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1084/1998] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:29 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:30.150147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-920f0f12-5f2b-45ed-820c-92376055697d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:30.150787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-920f0f12-5f2b-45ed-820c-92376055697d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:30.157733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-920f0f12-5f2b-45ed-820c-92376055697d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:30.158219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-920f0f12-5f2b-45ed-820c-92376055697d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:30.158969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-920f0f12-5f2b-45ed-820c-92376055697d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:30.159396 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-920f0f12-5f2b-45ed-820c-92376055697d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:30.219102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-920f0f12-5f2b-45ed-820c-92376055697d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:30.227468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-920f0f12-5f2b-45ed-820c-92376055697d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:30.228989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-920f0f12-5f2b-45ed-820c-92376055697d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:30.230128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-920f0f12-5f2b-45ed-820c-92376055697d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084568 Jun 10 13:27:30.230674 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1085/1999] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:30 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:31.043747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fafeacee-0975-4693-bc55-2d0686ca11a0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:31.044354 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fafeacee-0975-4693-bc55-2d0686ca11a0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:31.051246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fafeacee-0975-4693-bc55-2d0686ca11a0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:31.051689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fafeacee-0975-4693-bc55-2d0686ca11a0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:31.052434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fafeacee-0975-4693-bc55-2d0686ca11a0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:31.052889 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fafeacee-0975-4693-bc55-2d0686ca11a0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:31.177523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fafeacee-0975-4693-bc55-2d0686ca11a0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:31.185498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fafeacee-0975-4693-bc55-2d0686ca11a0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145647 Jun 10 13:27:31.187728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1086/2000] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:31 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:31.247464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-76a83ec7-b3fe-41fd-a32f-9c07128326c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:31.248068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-76a83ec7-b3fe-41fd-a32f-9c07128326c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:31.254033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76a83ec7-b3fe-41fd-a32f-9c07128326c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:31.256441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76a83ec7-b3fe-41fd-a32f-9c07128326c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:31.256441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76a83ec7-b3fe-41fd-a32f-9c07128326c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:31.256441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76a83ec7-b3fe-41fd-a32f-9c07128326c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:31.314300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-76a83ec7-b3fe-41fd-a32f-9c07128326c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:31.322860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-76a83ec7-b3fe-41fd-a32f-9c07128326c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:31.324367 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-76a83ec7-b3fe-41fd-a32f-9c07128326c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:31.325540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-76a83ec7-b3fe-41fd-a32f-9c07128326c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.081710 Jun 10 13:27:31.326103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1087/2001] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:31 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:27:32.202452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c6ebfb08-8fed-485c-a62c-7476d3e225a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:32.203079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c6ebfb08-8fed-485c-a62c-7476d3e225a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:32.209688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6ebfb08-8fed-485c-a62c-7476d3e225a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:32.210136 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6ebfb08-8fed-485c-a62c-7476d3e225a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:32.210887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6ebfb08-8fed-485c-a62c-7476d3e225a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:32.211321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6ebfb08-8fed-485c-a62c-7476d3e225a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:32.333709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c6ebfb08-8fed-485c-a62c-7476d3e225a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:32.341966 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c6ebfb08-8fed-485c-a62c-7476d3e225a8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143418 Jun 10 13:27:32.342528 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1088/2002] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:32 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:32.347411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a06acd27-6d2e-4b63-8822-b7d5cde5dc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:32.348005 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a06acd27-6d2e-4b63-8822-b7d5cde5dc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:32.353543 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a06acd27-6d2e-4b63-8822-b7d5cde5dc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:32.353981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a06acd27-6d2e-4b63-8822-b7d5cde5dc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:32.354726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a06acd27-6d2e-4b63-8822-b7d5cde5dc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:32.355150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a06acd27-6d2e-4b63-8822-b7d5cde5dc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:32.416264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a06acd27-6d2e-4b63-8822-b7d5cde5dc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:32.425248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a06acd27-6d2e-4b63-8822-b7d5cde5dc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:32.426728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a06acd27-6d2e-4b63-8822-b7d5cde5dc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:32.427848 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a06acd27-6d2e-4b63-8822-b7d5cde5dc1c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084396 Jun 10 13:27:32.428442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1089/2003] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:32 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:33.360191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b3c3e021-4c21-4ff8-8186-ce93a15b1537 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:33.361232 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b3c3e021-4c21-4ff8-8186-ce93a15b1537 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:33.368654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3c3e021-4c21-4ff8-8186-ce93a15b1537 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:33.369752 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3c3e021-4c21-4ff8-8186-ce93a15b1537 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:33.370713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3c3e021-4c21-4ff8-8186-ce93a15b1537 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:33.371233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b3c3e021-4c21-4ff8-8186-ce93a15b1537 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:33.500475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b3c3e021-4c21-4ff8-8186-ce93a15b1537 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:33.508383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b3c3e021-4c21-4ff8-8186-ce93a15b1537 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.152287 Jun 10 13:27:33.508972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1090/2004] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:33 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:33.513249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3f3ec632-2fc1-4cf0-b1e0-b279ca6902cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:33.513837 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3f3ec632-2fc1-4cf0-b1e0-b279ca6902cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:33.519804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f3ec632-2fc1-4cf0-b1e0-b279ca6902cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:33.520243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f3ec632-2fc1-4cf0-b1e0-b279ca6902cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:33.521014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f3ec632-2fc1-4cf0-b1e0-b279ca6902cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:33.521458 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f3ec632-2fc1-4cf0-b1e0-b279ca6902cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:33.580134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f3ec632-2fc1-4cf0-b1e0-b279ca6902cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:33.588416 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f3ec632-2fc1-4cf0-b1e0-b279ca6902cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:33.589863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f3ec632-2fc1-4cf0-b1e0-b279ca6902cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:33.591275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3f3ec632-2fc1-4cf0-b1e0-b279ca6902cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.081367 Jun 10 13:27:33.591821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1091/2005] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:33 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:34.525823 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a97303f7-b183-46a3-8d03-c1ca837afc06 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:34.526436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a97303f7-b183-46a3-8d03-c1ca837afc06 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:34.532996 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a97303f7-b183-46a3-8d03-c1ca837afc06 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:34.533448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a97303f7-b183-46a3-8d03-c1ca837afc06 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:34.534202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a97303f7-b183-46a3-8d03-c1ca837afc06 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:34.534638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a97303f7-b183-46a3-8d03-c1ca837afc06 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:34.665258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a97303f7-b183-46a3-8d03-c1ca837afc06 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:34.673040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a97303f7-b183-46a3-8d03-c1ca837afc06 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.151085 Jun 10 13:27:34.673602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1092/2006] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:34 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:27:34.677866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-be8a7397-8320-486b-8305-c5022593bdec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:34.678454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-be8a7397-8320-486b-8305-c5022593bdec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:34.683858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be8a7397-8320-486b-8305-c5022593bdec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:34.684299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be8a7397-8320-486b-8305-c5022593bdec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:34.685071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be8a7397-8320-486b-8305-c5022593bdec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:34.685505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be8a7397-8320-486b-8305-c5022593bdec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:34.744614 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-be8a7397-8320-486b-8305-c5022593bdec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:34.753744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-be8a7397-8320-486b-8305-c5022593bdec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:34.755226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-be8a7397-8320-486b-8305-c5022593bdec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:34.756370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-be8a7397-8320-486b-8305-c5022593bdec tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.081865 Jun 10 13:27:34.756947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1093/2007] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:34 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:35.690191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5e85db69-a2a3-4462-a50c-ccacf1530549 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:35.690843 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5e85db69-a2a3-4462-a50c-ccacf1530549 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:35.697169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e85db69-a2a3-4462-a50c-ccacf1530549 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:35.697620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e85db69-a2a3-4462-a50c-ccacf1530549 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:35.698433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e85db69-a2a3-4462-a50c-ccacf1530549 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:35.698868 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e85db69-a2a3-4462-a50c-ccacf1530549 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:35.823030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5e85db69-a2a3-4462-a50c-ccacf1530549 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:35.831500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5e85db69-a2a3-4462-a50c-ccacf1530549 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145243 Jun 10 13:27:35.832062 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1094/2008] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:35 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:35.836454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6dfc2957-f373-4db1-9a84-dbb8a26d6dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:35.837076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6dfc2957-f373-4db1-9a84-dbb8a26d6dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:35.842919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6dfc2957-f373-4db1-9a84-dbb8a26d6dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:35.843367 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6dfc2957-f373-4db1-9a84-dbb8a26d6dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:35.844687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6dfc2957-f373-4db1-9a84-dbb8a26d6dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:35.845146 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6dfc2957-f373-4db1-9a84-dbb8a26d6dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:35.905549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6dfc2957-f373-4db1-9a84-dbb8a26d6dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:35.914535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6dfc2957-f373-4db1-9a84-dbb8a26d6dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:35.916018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6dfc2957-f373-4db1-9a84-dbb8a26d6dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:35.917178 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6dfc2957-f373-4db1-9a84-dbb8a26d6dea tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084178 Jun 10 13:27:35.917739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1095/2009] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:35 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:36.849082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-23e86a86-6385-489f-b46b-50c9e9960e09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:36.849724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-23e86a86-6385-489f-b46b-50c9e9960e09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:36.856304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-23e86a86-6385-489f-b46b-50c9e9960e09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:36.856796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-23e86a86-6385-489f-b46b-50c9e9960e09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:36.857558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-23e86a86-6385-489f-b46b-50c9e9960e09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:36.857992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-23e86a86-6385-489f-b46b-50c9e9960e09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:36.876199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:27:36.983664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-23e86a86-6385-489f-b46b-50c9e9960e09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:36.992250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-23e86a86-6385-489f-b46b-50c9e9960e09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146988 Jun 10 13:27:36.992835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1096/2010] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:36 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:36.997187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-18554878-bc91-4cbf-80fd-20e24887036d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:36.997766 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-18554878-bc91-4cbf-80fd-20e24887036d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:37.003268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18554878-bc91-4cbf-80fd-20e24887036d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:37.003716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18554878-bc91-4cbf-80fd-20e24887036d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:37.004464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18554878-bc91-4cbf-80fd-20e24887036d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:37.004933 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18554878-bc91-4cbf-80fd-20e24887036d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:37.062979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-18554878-bc91-4cbf-80fd-20e24887036d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:37.071164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-18554878-bc91-4cbf-80fd-20e24887036d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:37.072657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-18554878-bc91-4cbf-80fd-20e24887036d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:37.073824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-18554878-bc91-4cbf-80fd-20e24887036d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.080075 Jun 10 13:27:37.074380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1097/2011] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:36 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:38.009398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5bb4f130-f88f-438e-9aa4-b893c47cd1ff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:38.010057 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5bb4f130-f88f-438e-9aa4-b893c47cd1ff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:38.012341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=91,nova_api:UPDATE=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:27:38.017112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5bb4f130-f88f-438e-9aa4-b893c47cd1ff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:38.017561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5bb4f130-f88f-438e-9aa4-b893c47cd1ff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:38.018321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5bb4f130-f88f-438e-9aa4-b893c47cd1ff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:38.018757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5bb4f130-f88f-438e-9aa4-b893c47cd1ff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:38.146374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5bb4f130-f88f-438e-9aa4-b893c47cd1ff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:38.154253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5bb4f130-f88f-438e-9aa4-b893c47cd1ff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148684 Jun 10 13:27:38.154821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1098/2012] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:38 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:38.159725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d8a4a258-e21e-45ba-945d-60b2754e40c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:38.160321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d8a4a258-e21e-45ba-945d-60b2754e40c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:38.165804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d8a4a258-e21e-45ba-945d-60b2754e40c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:38.166250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d8a4a258-e21e-45ba-945d-60b2754e40c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:38.167003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d8a4a258-e21e-45ba-945d-60b2754e40c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:38.167431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d8a4a258-e21e-45ba-945d-60b2754e40c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:38.225418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d8a4a258-e21e-45ba-945d-60b2754e40c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:38.233647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d8a4a258-e21e-45ba-945d-60b2754e40c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:38.235128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d8a4a258-e21e-45ba-945d-60b2754e40c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:38.236252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d8a4a258-e21e-45ba-945d-60b2754e40c2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.080531 Jun 10 13:27:38.236827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1099/2013] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:38 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:39.172462 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ff90e049-97dc-4e1b-9871-7bb80f5fb427 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:39.173166 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ff90e049-97dc-4e1b-9871-7bb80f5fb427 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:39.179569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff90e049-97dc-4e1b-9871-7bb80f5fb427 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:39.180015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff90e049-97dc-4e1b-9871-7bb80f5fb427 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:39.180787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff90e049-97dc-4e1b-9871-7bb80f5fb427 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:39.181233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff90e049-97dc-4e1b-9871-7bb80f5fb427 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:39.305262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ff90e049-97dc-4e1b-9871-7bb80f5fb427 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:39.313883 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ff90e049-97dc-4e1b-9871-7bb80f5fb427 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145908 Jun 10 13:27:39.314452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1100/2014] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:39 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:39.318821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ca1b7ff8-d348-4a86-8c09-92593ab3eee5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:39.319415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ca1b7ff8-d348-4a86-8c09-92593ab3eee5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:39.325345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca1b7ff8-d348-4a86-8c09-92593ab3eee5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:39.325788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca1b7ff8-d348-4a86-8c09-92593ab3eee5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:39.326540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca1b7ff8-d348-4a86-8c09-92593ab3eee5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:39.326970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca1b7ff8-d348-4a86-8c09-92593ab3eee5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:39.391418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ca1b7ff8-d348-4a86-8c09-92593ab3eee5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:39.399741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ca1b7ff8-d348-4a86-8c09-92593ab3eee5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:39.401267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ca1b7ff8-d348-4a86-8c09-92593ab3eee5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:39.402392 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ca1b7ff8-d348-4a86-8c09-92593ab3eee5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087045 Jun 10 13:27:39.402945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1101/2015] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:39 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:40.331473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-93de52b0-3045-4dbc-ae95-9430d3d95680 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:40.332206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-93de52b0-3045-4dbc-ae95-9430d3d95680 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:40.341193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-93de52b0-3045-4dbc-ae95-9430d3d95680 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:40.341637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-93de52b0-3045-4dbc-ae95-9430d3d95680 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:40.342900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-93de52b0-3045-4dbc-ae95-9430d3d95680 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:40.344297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-93de52b0-3045-4dbc-ae95-9430d3d95680 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:40.469287 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-93de52b0-3045-4dbc-ae95-9430d3d95680 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:40.477090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-93de52b0-3045-4dbc-ae95-9430d3d95680 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150024 Jun 10 13:27:40.477652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1102/2016] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:40 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:40.482006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1a8ddd46-d8cd-4d36-97fb-911d4c178745 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:40.482594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1a8ddd46-d8cd-4d36-97fb-911d4c178745 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:40.488873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a8ddd46-d8cd-4d36-97fb-911d4c178745 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:40.489322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a8ddd46-d8cd-4d36-97fb-911d4c178745 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:40.490071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a8ddd46-d8cd-4d36-97fb-911d4c178745 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:40.490497 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a8ddd46-d8cd-4d36-97fb-911d4c178745 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:40.548875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1a8ddd46-d8cd-4d36-97fb-911d4c178745 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:40.557280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1a8ddd46-d8cd-4d36-97fb-911d4c178745 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:40.558794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1a8ddd46-d8cd-4d36-97fb-911d4c178745 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:40.559936 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1a8ddd46-d8cd-4d36-97fb-911d4c178745 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.081372 Jun 10 13:27:40.560491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1103/2017] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:40 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:41.493935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-694b977e-eb60-4a35-a78d-d90e3780f6cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:41.494597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-694b977e-eb60-4a35-a78d-d90e3780f6cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:41.501158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-694b977e-eb60-4a35-a78d-d90e3780f6cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:41.508042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-694b977e-eb60-4a35-a78d-d90e3780f6cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:41.508042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-694b977e-eb60-4a35-a78d-d90e3780f6cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:41.508042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-694b977e-eb60-4a35-a78d-d90e3780f6cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:41.633789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-694b977e-eb60-4a35-a78d-d90e3780f6cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:41.642275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-694b977e-eb60-4a35-a78d-d90e3780f6cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.152197 Jun 10 13:27:41.642838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1104/2018] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:41 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:41.647215 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d5559595-0ae2-4e12-a210-ba45818947fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:41.647815 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d5559595-0ae2-4e12-a210-ba45818947fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:41.653453 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5559595-0ae2-4e12-a210-ba45818947fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:41.653903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5559595-0ae2-4e12-a210-ba45818947fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:41.654660 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5559595-0ae2-4e12-a210-ba45818947fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:41.655097 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5559595-0ae2-4e12-a210-ba45818947fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:41.713529 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d5559595-0ae2-4e12-a210-ba45818947fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:41.722575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d5559595-0ae2-4e12-a210-ba45818947fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:41.724074 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d5559595-0ae2-4e12-a210-ba45818947fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:41.725336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d5559595-0ae2-4e12-a210-ba45818947fb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.081584 Jun 10 13:27:41.725894 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1105/2019] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:41 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:42.658098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e4716076-d2af-4d76-880d-5ab3260b40c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:42.658736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e4716076-d2af-4d76-880d-5ab3260b40c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:42.665176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4716076-d2af-4d76-880d-5ab3260b40c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:42.665632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4716076-d2af-4d76-880d-5ab3260b40c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:42.666391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4716076-d2af-4d76-880d-5ab3260b40c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:42.666830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4716076-d2af-4d76-880d-5ab3260b40c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:42.790566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e4716076-d2af-4d76-880d-5ab3260b40c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:42.799117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e4716076-d2af-4d76-880d-5ab3260b40c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144930 Jun 10 13:27:42.799691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1106/2020] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:42 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:42.804029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8f2fe5ea-731b-415d-94db-45e51e03260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:42.804629 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8f2fe5ea-731b-415d-94db-45e51e03260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:42.810199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f2fe5ea-731b-415d-94db-45e51e03260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:42.810670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f2fe5ea-731b-415d-94db-45e51e03260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:42.811428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f2fe5ea-731b-415d-94db-45e51e03260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:42.811866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f2fe5ea-731b-415d-94db-45e51e03260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:42.873536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8f2fe5ea-731b-415d-94db-45e51e03260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:42.882038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8f2fe5ea-731b-415d-94db-45e51e03260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:42.883545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8f2fe5ea-731b-415d-94db-45e51e03260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:42.884681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8f2fe5ea-731b-415d-94db-45e51e03260f tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084087 Jun 10 13:27:42.885280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1107/2021] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:42 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:43.816040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2e86afe9-fe00-4914-bc68-bdcc079560f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:43.816698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2e86afe9-fe00-4914-bc68-bdcc079560f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:43.823110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e86afe9-fe00-4914-bc68-bdcc079560f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:43.823553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e86afe9-fe00-4914-bc68-bdcc079560f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:43.824296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e86afe9-fe00-4914-bc68-bdcc079560f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:43.824724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e86afe9-fe00-4914-bc68-bdcc079560f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:43.948563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2e86afe9-fe00-4914-bc68-bdcc079560f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:43.956964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2e86afe9-fe00-4914-bc68-bdcc079560f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144781 Jun 10 13:27:43.957527 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1108/2022] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:43 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:43.961848 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ee060e9d-a27a-4bb7-b89c-1a03551b58c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:43.962565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ee060e9d-a27a-4bb7-b89c-1a03551b58c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:43.968217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee060e9d-a27a-4bb7-b89c-1a03551b58c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:43.968653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee060e9d-a27a-4bb7-b89c-1a03551b58c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:43.969425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee060e9d-a27a-4bb7-b89c-1a03551b58c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:43.970471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ee060e9d-a27a-4bb7-b89c-1a03551b58c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:44.029797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ee060e9d-a27a-4bb7-b89c-1a03551b58c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:44.038671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ee060e9d-a27a-4bb7-b89c-1a03551b58c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:44.040154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ee060e9d-a27a-4bb7-b89c-1a03551b58c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:44.041313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ee060e9d-a27a-4bb7-b89c-1a03551b58c0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082881 Jun 10 13:27:44.041859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1109/2023] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:43 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:44.974298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5aa8ca41-e42d-46df-869c-caa420f1a38f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:44.974943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5aa8ca41-e42d-46df-869c-caa420f1a38f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:44.981469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5aa8ca41-e42d-46df-869c-caa420f1a38f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:44.981934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5aa8ca41-e42d-46df-869c-caa420f1a38f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:44.982688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5aa8ca41-e42d-46df-869c-caa420f1a38f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:44.983121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5aa8ca41-e42d-46df-869c-caa420f1a38f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:45.107494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5aa8ca41-e42d-46df-869c-caa420f1a38f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:45.115391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5aa8ca41-e42d-46df-869c-caa420f1a38f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144963 Jun 10 13:27:45.115956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1110/2024] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:44 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:45.120345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-84f3b33b-46ee-4f46-93a5-ee65c69ef712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:45.120972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-84f3b33b-46ee-4f46-93a5-ee65c69ef712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:45.127052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84f3b33b-46ee-4f46-93a5-ee65c69ef712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:45.127494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84f3b33b-46ee-4f46-93a5-ee65c69ef712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:45.128245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84f3b33b-46ee-4f46-93a5-ee65c69ef712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:45.128678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-84f3b33b-46ee-4f46-93a5-ee65c69ef712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:45.185803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-84f3b33b-46ee-4f46-93a5-ee65c69ef712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:45.194492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-84f3b33b-46ee-4f46-93a5-ee65c69ef712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:45.195988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-84f3b33b-46ee-4f46-93a5-ee65c69ef712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:45.197143 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-84f3b33b-46ee-4f46-93a5-ee65c69ef712 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.080248 Jun 10 13:27:45.197696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1111/2025] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:45 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:46.132227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d09a9c48-c536-43ea-ac69-2815fc93efd9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:46.132888 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d09a9c48-c536-43ea-ac69-2815fc93efd9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:46.139283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d09a9c48-c536-43ea-ac69-2815fc93efd9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:46.139728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d09a9c48-c536-43ea-ac69-2815fc93efd9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:46.140478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d09a9c48-c536-43ea-ac69-2815fc93efd9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:46.140945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d09a9c48-c536-43ea-ac69-2815fc93efd9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:46.264266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d09a9c48-c536-43ea-ac69-2815fc93efd9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:46.272680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d09a9c48-c536-43ea-ac69-2815fc93efd9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144396 Jun 10 13:27:46.273274 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1112/2026] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:46 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:46.277749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-15702bd7-3b54-40ac-ba07-e5559181991d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:46.278348 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-15702bd7-3b54-40ac-ba07-e5559181991d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:46.284452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15702bd7-3b54-40ac-ba07-e5559181991d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:46.284927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15702bd7-3b54-40ac-ba07-e5559181991d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:46.285684 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15702bd7-3b54-40ac-ba07-e5559181991d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:46.286114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15702bd7-3b54-40ac-ba07-e5559181991d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:46.290013 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:27:46.349175 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-15702bd7-3b54-40ac-ba07-e5559181991d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:46.357718 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-15702bd7-3b54-40ac-ba07-e5559181991d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:46.359238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-15702bd7-3b54-40ac-ba07-e5559181991d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:46.360374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-15702bd7-3b54-40ac-ba07-e5559181991d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086189 Jun 10 13:27:46.360960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1113/2027] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:46 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:47.289535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ceff0e07-1e22-4e76-99c1-f0c600a6053e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:47.290212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ceff0e07-1e22-4e76-99c1-f0c600a6053e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:47.296664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ceff0e07-1e22-4e76-99c1-f0c600a6053e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:47.301067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ceff0e07-1e22-4e76-99c1-f0c600a6053e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:47.301067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ceff0e07-1e22-4e76-99c1-f0c600a6053e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:47.301067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ceff0e07-1e22-4e76-99c1-f0c600a6053e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:47.423267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ceff0e07-1e22-4e76-99c1-f0c600a6053e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:47.431146 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ceff0e07-1e22-4e76-99c1-f0c600a6053e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145478 Jun 10 13:27:47.431714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1114/2028] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:47 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:47.436190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-95bb04b9-b070-4f34-a8fe-19823f730d90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:47.436797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-95bb04b9-b070-4f34-a8fe-19823f730d90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:47.442574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95bb04b9-b070-4f34-a8fe-19823f730d90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:47.443026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95bb04b9-b070-4f34-a8fe-19823f730d90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:47.443791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95bb04b9-b070-4f34-a8fe-19823f730d90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:47.444226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95bb04b9-b070-4f34-a8fe-19823f730d90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:47.502623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-95bb04b9-b070-4f34-a8fe-19823f730d90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:47.510811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-95bb04b9-b070-4f34-a8fe-19823f730d90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:47.512313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-95bb04b9-b070-4f34-a8fe-19823f730d90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:47.513478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-95bb04b9-b070-4f34-a8fe-19823f730d90 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.080855 Jun 10 13:27:47.514039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1115/2029] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:47 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:48.448047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d7640dbd-170f-4e8f-bc30-579da3687917 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:48.448649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d7640dbd-170f-4e8f-bc30-579da3687917 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:48.455635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d7640dbd-170f-4e8f-bc30-579da3687917 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:48.456074 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d7640dbd-170f-4e8f-bc30-579da3687917 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:48.456842 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d7640dbd-170f-4e8f-bc30-579da3687917 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:48.457270 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d7640dbd-170f-4e8f-bc30-579da3687917 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:48.586414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d7640dbd-170f-4e8f-bc30-579da3687917 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:48.595063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d7640dbd-170f-4e8f-bc30-579da3687917 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150279 Jun 10 13:27:48.595638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1116/2030] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:48 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:48.599968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-34dc7b0a-33d5-4fc8-9939-baf0d7fbc11c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:48.600565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-34dc7b0a-33d5-4fc8-9939-baf0d7fbc11c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:48.606082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-34dc7b0a-33d5-4fc8-9939-baf0d7fbc11c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:48.606536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-34dc7b0a-33d5-4fc8-9939-baf0d7fbc11c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:48.607287 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-34dc7b0a-33d5-4fc8-9939-baf0d7fbc11c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:48.607718 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-34dc7b0a-33d5-4fc8-9939-baf0d7fbc11c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:48.666063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-34dc7b0a-33d5-4fc8-9939-baf0d7fbc11c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:48.674252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-34dc7b0a-33d5-4fc8-9939-baf0d7fbc11c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:48.675729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-34dc7b0a-33d5-4fc8-9939-baf0d7fbc11c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:48.676892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-34dc7b0a-33d5-4fc8-9939-baf0d7fbc11c tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.080326 Jun 10 13:27:48.677451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1117/2031] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:48 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:49.612834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-97779248-e21f-4aca-8065-9818c1611961 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:49.612834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-97779248-e21f-4aca-8065-9818c1611961 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:49.619894 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97779248-e21f-4aca-8065-9818c1611961 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:49.620362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97779248-e21f-4aca-8065-9818c1611961 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:49.621220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97779248-e21f-4aca-8065-9818c1611961 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:49.621654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-97779248-e21f-4aca-8065-9818c1611961 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:49.749716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-97779248-e21f-4aca-8065-9818c1611961 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:49.758239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-97779248-e21f-4aca-8065-9818c1611961 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150092 Jun 10 13:27:49.758820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1118/2032] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:49 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:49.763254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2f7ffe2e-30a2-4a59-a327-04da354c0e09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:49.763848 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2f7ffe2e-30a2-4a59-a327-04da354c0e09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:49.770187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f7ffe2e-30a2-4a59-a327-04da354c0e09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:49.770635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f7ffe2e-30a2-4a59-a327-04da354c0e09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:49.771391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f7ffe2e-30a2-4a59-a327-04da354c0e09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:49.771828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f7ffe2e-30a2-4a59-a327-04da354c0e09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:49.833825 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2f7ffe2e-30a2-4a59-a327-04da354c0e09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:49.842344 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2f7ffe2e-30a2-4a59-a327-04da354c0e09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:49.843836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2f7ffe2e-30a2-4a59-a327-04da354c0e09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:49.845026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2f7ffe2e-30a2-4a59-a327-04da354c0e09 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085262 Jun 10 13:27:49.845584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1119/2033] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:49 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:50.775526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-351dcb4d-d833-4ccd-9014-7abfd1abd760 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:50.776164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-351dcb4d-d833-4ccd-9014-7abfd1abd760 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:50.782595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-351dcb4d-d833-4ccd-9014-7abfd1abd760 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:50.783042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-351dcb4d-d833-4ccd-9014-7abfd1abd760 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:50.783793 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-351dcb4d-d833-4ccd-9014-7abfd1abd760 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:50.784228 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-351dcb4d-d833-4ccd-9014-7abfd1abd760 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:51.087632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-351dcb4d-d833-4ccd-9014-7abfd1abd760 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:51.096240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-351dcb4d-d833-4ccd-9014-7abfd1abd760 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.324697 Jun 10 13:27:51.096823 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1120/2034] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:50 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:51.101137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c54d7c4e-240e-4d50-8619-1c507851f819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:51.101736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c54d7c4e-240e-4d50-8619-1c507851f819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:51.107176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c54d7c4e-240e-4d50-8619-1c507851f819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:51.107627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c54d7c4e-240e-4d50-8619-1c507851f819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:51.108377 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c54d7c4e-240e-4d50-8619-1c507851f819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:51.108838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c54d7c4e-240e-4d50-8619-1c507851f819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:51.166728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c54d7c4e-240e-4d50-8619-1c507851f819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:51.175729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c54d7c4e-240e-4d50-8619-1c507851f819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:51.177236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c54d7c4e-240e-4d50-8619-1c507851f819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:51.178364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c54d7c4e-240e-4d50-8619-1c507851f819 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.080633 Jun 10 13:27:51.178956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1121/2035] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:51 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:52.113348 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a29e949c-c204-4213-a62e-031c082dd4de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:52.114037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a29e949c-c204-4213-a62e-031c082dd4de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:52.120590 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a29e949c-c204-4213-a62e-031c082dd4de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:52.121062 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a29e949c-c204-4213-a62e-031c082dd4de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:52.121813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a29e949c-c204-4213-a62e-031c082dd4de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:52.122242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a29e949c-c204-4213-a62e-031c082dd4de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:52.247473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a29e949c-c204-4213-a62e-031c082dd4de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:52.255402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a29e949c-c204-4213-a62e-031c082dd4de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146038 Jun 10 13:27:52.255966 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1122/2036] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:52 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:52.260291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a7f1a170-508f-4aa1-bc65-54af4c1f01c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:52.260903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a7f1a170-508f-4aa1-bc65-54af4c1f01c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:52.267190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7f1a170-508f-4aa1-bc65-54af4c1f01c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:52.267641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7f1a170-508f-4aa1-bc65-54af4c1f01c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:52.268429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7f1a170-508f-4aa1-bc65-54af4c1f01c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:52.268893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7f1a170-508f-4aa1-bc65-54af4c1f01c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:52.326902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a7f1a170-508f-4aa1-bc65-54af4c1f01c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:52.335789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a7f1a170-508f-4aa1-bc65-54af4c1f01c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:52.337317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a7f1a170-508f-4aa1-bc65-54af4c1f01c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:52.338461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a7f1a170-508f-4aa1-bc65-54af4c1f01c8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.081586 Jun 10 13:27:52.339023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1123/2037] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:52 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:53.272142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1b2160c1-7681-4ba8-897b-9d7f344dc177 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:53.272852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1b2160c1-7681-4ba8-897b-9d7f344dc177 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:53.279391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b2160c1-7681-4ba8-897b-9d7f344dc177 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:53.279836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b2160c1-7681-4ba8-897b-9d7f344dc177 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:53.280588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b2160c1-7681-4ba8-897b-9d7f344dc177 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:53.281053 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b2160c1-7681-4ba8-897b-9d7f344dc177 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:53.407599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1b2160c1-7681-4ba8-897b-9d7f344dc177 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:53.416066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1b2160c1-7681-4ba8-897b-9d7f344dc177 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147798 Jun 10 13:27:53.416629 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1124/2038] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:53 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:53.421021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0ae551c8-fd41-4b25-9408-32e5d892a698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:53.421617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0ae551c8-fd41-4b25-9408-32e5d892a698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:53.427576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ae551c8-fd41-4b25-9408-32e5d892a698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:53.428021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ae551c8-fd41-4b25-9408-32e5d892a698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:53.428790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ae551c8-fd41-4b25-9408-32e5d892a698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:53.429233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ae551c8-fd41-4b25-9408-32e5d892a698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:53.490102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0ae551c8-fd41-4b25-9408-32e5d892a698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:53.498316 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0ae551c8-fd41-4b25-9408-32e5d892a698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:53.499790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0ae551c8-fd41-4b25-9408-32e5d892a698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:53.500953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0ae551c8-fd41-4b25-9408-32e5d892a698 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083355 Jun 10 13:27:53.502023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1125/2039] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:53 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:54.433083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-39ee473e-aae6-4fba-a09d-8c82a128086f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:54.433673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-39ee473e-aae6-4fba-a09d-8c82a128086f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:54.440844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39ee473e-aae6-4fba-a09d-8c82a128086f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:54.441329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39ee473e-aae6-4fba-a09d-8c82a128086f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:54.442086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39ee473e-aae6-4fba-a09d-8c82a128086f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:54.442520 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39ee473e-aae6-4fba-a09d-8c82a128086f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:54.567131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-39ee473e-aae6-4fba-a09d-8c82a128086f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:54.575017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-39ee473e-aae6-4fba-a09d-8c82a128086f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145883 Jun 10 13:27:54.575587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1126/2040] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:54 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:54.579965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-60159dd1-e7ed-4ced-aa71-98dac3c232e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:54.580564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-60159dd1-e7ed-4ced-aa71-98dac3c232e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:54.586232 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60159dd1-e7ed-4ced-aa71-98dac3c232e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:54.586620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60159dd1-e7ed-4ced-aa71-98dac3c232e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:54.587373 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60159dd1-e7ed-4ced-aa71-98dac3c232e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:54.587807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-60159dd1-e7ed-4ced-aa71-98dac3c232e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:54.646000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-60159dd1-e7ed-4ced-aa71-98dac3c232e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:54.654126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-60159dd1-e7ed-4ced-aa71-98dac3c232e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:54.655618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-60159dd1-e7ed-4ced-aa71-98dac3c232e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:54.656743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-60159dd1-e7ed-4ced-aa71-98dac3c232e0 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.080250 Jun 10 13:27:54.657320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1127/2041] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:54 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:55.592109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-25e2bdb0-5b2a-4e45-89a6-ad3a403acf09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:55.592694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-25e2bdb0-5b2a-4e45-89a6-ad3a403acf09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:55.599126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25e2bdb0-5b2a-4e45-89a6-ad3a403acf09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:55.599574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25e2bdb0-5b2a-4e45-89a6-ad3a403acf09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:55.600325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25e2bdb0-5b2a-4e45-89a6-ad3a403acf09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:55.600757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25e2bdb0-5b2a-4e45-89a6-ad3a403acf09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:55.728723 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-25e2bdb0-5b2a-4e45-89a6-ad3a403acf09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:55.737168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-25e2bdb0-5b2a-4e45-89a6-ad3a403acf09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149232 Jun 10 13:27:55.737734 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1128/2042] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:55 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:55.742088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c073e492-a4b5-4b9e-9e59-e442fa1e185d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:55.742680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c073e492-a4b5-4b9e-9e59-e442fa1e185d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:55.748070 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c073e492-a4b5-4b9e-9e59-e442fa1e185d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:55.748508 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c073e492-a4b5-4b9e-9e59-e442fa1e185d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:55.749282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c073e492-a4b5-4b9e-9e59-e442fa1e185d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:55.749711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c073e492-a4b5-4b9e-9e59-e442fa1e185d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:55.808789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c073e492-a4b5-4b9e-9e59-e442fa1e185d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:55.812056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:27:55.817233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c073e492-a4b5-4b9e-9e59-e442fa1e185d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:55.818715 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c073e492-a4b5-4b9e-9e59-e442fa1e185d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:55.819829 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c073e492-a4b5-4b9e-9e59-e442fa1e185d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.081200 Jun 10 13:27:55.820382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1129/2043] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:55 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:56.754599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c8e196a1-6c80-4ee7-b094-d54a55a3e572 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:56.755266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c8e196a1-6c80-4ee7-b094-d54a55a3e572 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:56.762118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c8e196a1-6c80-4ee7-b094-d54a55a3e572 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:56.762556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c8e196a1-6c80-4ee7-b094-d54a55a3e572 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:56.763298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c8e196a1-6c80-4ee7-b094-d54a55a3e572 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:56.763726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c8e196a1-6c80-4ee7-b094-d54a55a3e572 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:56.887495 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c8e196a1-6c80-4ee7-b094-d54a55a3e572 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:56.895529 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c8e196a1-6c80-4ee7-b094-d54a55a3e572 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145368 Jun 10 13:27:56.896110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1130/2044] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:56 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:56.900528 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-636f8225-e9ee-49d4-aa7c-bb936312aa02 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:56.901155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-636f8225-e9ee-49d4-aa7c-bb936312aa02 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:56.906935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-636f8225-e9ee-49d4-aa7c-bb936312aa02 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:56.907378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-636f8225-e9ee-49d4-aa7c-bb936312aa02 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:56.908789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-636f8225-e9ee-49d4-aa7c-bb936312aa02 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:56.909227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-636f8225-e9ee-49d4-aa7c-bb936312aa02 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:56.973096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-636f8225-e9ee-49d4-aa7c-bb936312aa02 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:56.981356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-636f8225-e9ee-49d4-aa7c-bb936312aa02 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:56.982850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-636f8225-e9ee-49d4-aa7c-bb936312aa02 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:56.983981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-636f8225-e9ee-49d4-aa7c-bb936312aa02 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.086967 Jun 10 13:27:56.984539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1131/2045] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:56 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:57.913276 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-64581445-1fb1-474d-b546-d7b3d6285604 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:57.913946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-64581445-1fb1-474d-b546-d7b3d6285604 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:57.920536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-64581445-1fb1-474d-b546-d7b3d6285604 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:57.921061 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-64581445-1fb1-474d-b546-d7b3d6285604 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:57.921810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-64581445-1fb1-474d-b546-d7b3d6285604 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:57.922238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-64581445-1fb1-474d-b546-d7b3d6285604 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:58.055610 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-64581445-1fb1-474d-b546-d7b3d6285604 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:58.063610 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-64581445-1fb1-474d-b546-d7b3d6285604 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.154183 Jun 10 13:27:58.064856 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1132/2046] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:57 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:58.069347 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-70b56f71-35df-4e19-9943-b3ae1da2bbe6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:58.069944 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-70b56f71-35df-4e19-9943-b3ae1da2bbe6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:58.075443 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-70b56f71-35df-4e19-9943-b3ae1da2bbe6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:58.075886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-70b56f71-35df-4e19-9943-b3ae1da2bbe6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:58.076634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-70b56f71-35df-4e19-9943-b3ae1da2bbe6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:58.077097 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-70b56f71-35df-4e19-9943-b3ae1da2bbe6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:58.137875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-70b56f71-35df-4e19-9943-b3ae1da2bbe6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:58.146367 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-70b56f71-35df-4e19-9943-b3ae1da2bbe6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:58.147852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-70b56f71-35df-4e19-9943-b3ae1da2bbe6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:58.149026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-70b56f71-35df-4e19-9943-b3ae1da2bbe6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083226 Jun 10 13:27:58.149593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1133/2047] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:58 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:27:59.081735 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f629b5fd-0068-4653-afc6-3955349eae70 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:59.082373 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f629b5fd-0068-4653-afc6-3955349eae70 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:59.088804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f629b5fd-0068-4653-afc6-3955349eae70 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:59.089255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f629b5fd-0068-4653-afc6-3955349eae70 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:59.090271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f629b5fd-0068-4653-afc6-3955349eae70 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:59.090701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f629b5fd-0068-4653-afc6-3955349eae70 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:59.217715 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f629b5fd-0068-4653-afc6-3955349eae70 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:59.225581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f629b5fd-0068-4653-afc6-3955349eae70 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148326 Jun 10 13:27:59.226135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1134/2048] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:27:59 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:27:59.230532 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6e85244f-b434-472d-ba68-fc2899731332 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:27:59.231130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6e85244f-b434-472d-ba68-fc2899731332 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:27:59.236929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e85244f-b434-472d-ba68-fc2899731332 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:59.237384 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e85244f-b434-472d-ba68-fc2899731332 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:59.238176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e85244f-b434-472d-ba68-fc2899731332 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:27:59.238616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e85244f-b434-472d-ba68-fc2899731332 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:27:59.298377 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6e85244f-b434-472d-ba68-fc2899731332 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:59.306697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6e85244f-b434-472d-ba68-fc2899731332 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:59.308192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6e85244f-b434-472d-ba68-fc2899731332 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:27:59.309360 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6e85244f-b434-472d-ba68-fc2899731332 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082317 Jun 10 13:27:59.309917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1135/2049] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:27:59 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:00.242507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fceaa35f-63b7-4ea3-8fb6-c307dc741aea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:00.243204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fceaa35f-63b7-4ea3-8fb6-c307dc741aea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:00.249665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fceaa35f-63b7-4ea3-8fb6-c307dc741aea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:00.250113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fceaa35f-63b7-4ea3-8fb6-c307dc741aea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:00.250901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fceaa35f-63b7-4ea3-8fb6-c307dc741aea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:00.251845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fceaa35f-63b7-4ea3-8fb6-c307dc741aea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:00.375866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fceaa35f-63b7-4ea3-8fb6-c307dc741aea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:00.384485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fceaa35f-63b7-4ea3-8fb6-c307dc741aea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145813 Jun 10 13:28:00.385097 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1136/2050] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:00 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:00.389432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-01573818-e0b0-4c23-9d96-324ca7f1a67e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:00.390014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-01573818-e0b0-4c23-9d96-324ca7f1a67e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:00.395499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01573818-e0b0-4c23-9d96-324ca7f1a67e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:00.395943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01573818-e0b0-4c23-9d96-324ca7f1a67e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:00.396695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01573818-e0b0-4c23-9d96-324ca7f1a67e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:00.397151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01573818-e0b0-4c23-9d96-324ca7f1a67e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:00.458555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-01573818-e0b0-4c23-9d96-324ca7f1a67e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:00.467122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-01573818-e0b0-4c23-9d96-324ca7f1a67e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:00.468602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-01573818-e0b0-4c23-9d96-324ca7f1a67e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:00.469755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-01573818-e0b0-4c23-9d96-324ca7f1a67e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083756 Jun 10 13:28:00.470314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1137/2051] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:00 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:01.401142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-38f7462f-8f11-48d9-aaab-0331372b5c53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:01.401783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-38f7462f-8f11-48d9-aaab-0331372b5c53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:01.408532 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-38f7462f-8f11-48d9-aaab-0331372b5c53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:01.409010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-38f7462f-8f11-48d9-aaab-0331372b5c53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:01.409767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-38f7462f-8f11-48d9-aaab-0331372b5c53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:01.410199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-38f7462f-8f11-48d9-aaab-0331372b5c53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:01.534736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-38f7462f-8f11-48d9-aaab-0331372b5c53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:01.542676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-38f7462f-8f11-48d9-aaab-0331372b5c53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145556 Jun 10 13:28:01.543238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1138/2052] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:01 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:01.547574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-49693f65-f947-47f5-8134-d4cfc8e83c43 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:01.548160 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-49693f65-f947-47f5-8134-d4cfc8e83c43 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:01.553703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-49693f65-f947-47f5-8134-d4cfc8e83c43 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:01.554151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-49693f65-f947-47f5-8134-d4cfc8e83c43 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:01.554899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-49693f65-f947-47f5-8134-d4cfc8e83c43 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:01.555329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-49693f65-f947-47f5-8134-d4cfc8e83c43 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:01.613801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-49693f65-f947-47f5-8134-d4cfc8e83c43 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:01.622013 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-49693f65-f947-47f5-8134-d4cfc8e83c43 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:01.623516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-49693f65-f947-47f5-8134-d4cfc8e83c43 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:01.624641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-49693f65-f947-47f5-8134-d4cfc8e83c43 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.080499 Jun 10 13:28:01.625232 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1139/2053] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:01 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:02.559570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a82f4bc4-9e7b-4eb2-8479-9e710167ba08 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:02.560212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a82f4bc4-9e7b-4eb2-8479-9e710167ba08 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:02.566909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a82f4bc4-9e7b-4eb2-8479-9e710167ba08 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:02.567354 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a82f4bc4-9e7b-4eb2-8479-9e710167ba08 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:02.568105 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a82f4bc4-9e7b-4eb2-8479-9e710167ba08 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:02.568539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a82f4bc4-9e7b-4eb2-8479-9e710167ba08 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:02.699844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a82f4bc4-9e7b-4eb2-8479-9e710167ba08 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:02.708200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a82f4bc4-9e7b-4eb2-8479-9e710167ba08 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.152472 Jun 10 13:28:02.708792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1140/2054] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:02 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:02.713176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-df9b48c5-f415-4f24-9a19-a8f5baadde64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:02.713772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-df9b48c5-f415-4f24-9a19-a8f5baadde64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:02.719469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df9b48c5-f415-4f24-9a19-a8f5baadde64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:02.720545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df9b48c5-f415-4f24-9a19-a8f5baadde64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:02.721315 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df9b48c5-f415-4f24-9a19-a8f5baadde64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:02.721745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df9b48c5-f415-4f24-9a19-a8f5baadde64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:02.780623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-df9b48c5-f415-4f24-9a19-a8f5baadde64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:02.789599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-df9b48c5-f415-4f24-9a19-a8f5baadde64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:02.791084 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-df9b48c5-f415-4f24-9a19-a8f5baadde64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:02.792204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-df9b48c5-f415-4f24-9a19-a8f5baadde64 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082526 Jun 10 13:28:02.792784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1141/2055] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:02 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:03.725066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c002ad31-b4c0-461a-b7e8-81a22fa9feed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:03.725712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c002ad31-b4c0-461a-b7e8-81a22fa9feed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:03.732217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c002ad31-b4c0-461a-b7e8-81a22fa9feed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:03.732662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c002ad31-b4c0-461a-b7e8-81a22fa9feed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:03.733445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c002ad31-b4c0-461a-b7e8-81a22fa9feed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:03.733879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c002ad31-b4c0-461a-b7e8-81a22fa9feed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:03.855743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c002ad31-b4c0-461a-b7e8-81a22fa9feed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:03.863627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c002ad31-b4c0-461a-b7e8-81a22fa9feed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.142450 Jun 10 13:28:03.864192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1142/2056] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:03 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:03.868504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b63a787e-7c4a-43fe-b7c6-9f542224cbb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:03.869128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b63a787e-7c4a-43fe-b7c6-9f542224cbb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:03.874606 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b63a787e-7c4a-43fe-b7c6-9f542224cbb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:03.875048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b63a787e-7c4a-43fe-b7c6-9f542224cbb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:03.875796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b63a787e-7c4a-43fe-b7c6-9f542224cbb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:03.876737 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b63a787e-7c4a-43fe-b7c6-9f542224cbb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:03.937337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b63a787e-7c4a-43fe-b7c6-9f542224cbb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:03.946183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b63a787e-7c4a-43fe-b7c6-9f542224cbb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:03.947676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b63a787e-7c4a-43fe-b7c6-9f542224cbb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:03.948835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b63a787e-7c4a-43fe-b7c6-9f542224cbb4 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083708 Jun 10 13:28:03.949398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1143/2057] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:03 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:04.881371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5b4a7d83-6d2f-4107-91dc-dbcb41fcd3f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:04.882093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5b4a7d83-6d2f-4107-91dc-dbcb41fcd3f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:04.888478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5b4a7d83-6d2f-4107-91dc-dbcb41fcd3f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:04.888965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5b4a7d83-6d2f-4107-91dc-dbcb41fcd3f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:04.889760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5b4a7d83-6d2f-4107-91dc-dbcb41fcd3f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:04.890195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5b4a7d83-6d2f-4107-91dc-dbcb41fcd3f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:05.015439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5b4a7d83-6d2f-4107-91dc-dbcb41fcd3f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:05.027011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5b4a7d83-6d2f-4107-91dc-dbcb41fcd3f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149546 Jun 10 13:28:05.027593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1144/2058] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:04 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:05.032410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b8939e03-3c58-4fc0-8403-baf3135c14b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:05.033105 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b8939e03-3c58-4fc0-8403-baf3135c14b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:05.039627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8939e03-3c58-4fc0-8403-baf3135c14b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:05.040075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8939e03-3c58-4fc0-8403-baf3135c14b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:05.040857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8939e03-3c58-4fc0-8403-baf3135c14b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:05.041288 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b8939e03-3c58-4fc0-8403-baf3135c14b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:05.100619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b8939e03-3c58-4fc0-8403-baf3135c14b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:05.108868 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b8939e03-3c58-4fc0-8403-baf3135c14b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:05.110911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b8939e03-3c58-4fc0-8403-baf3135c14b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:05.112026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b8939e03-3c58-4fc0-8403-baf3135c14b1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083520 Jun 10 13:28:05.112576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1145/2059] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:05 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:06.046481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7b1ec24c-b0b9-413d-87ed-aea7827a7592 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:06.047158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7b1ec24c-b0b9-413d-87ed-aea7827a7592 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:06.054263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b1ec24c-b0b9-413d-87ed-aea7827a7592 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:06.054717 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b1ec24c-b0b9-413d-87ed-aea7827a7592 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:06.055474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b1ec24c-b0b9-413d-87ed-aea7827a7592 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:06.055906 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7b1ec24c-b0b9-413d-87ed-aea7827a7592 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:06.073285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:28:06.184328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7b1ec24c-b0b9-413d-87ed-aea7827a7592 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:06.192444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7b1ec24c-b0b9-413d-87ed-aea7827a7592 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149830 Jun 10 13:28:06.193047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1146/2060] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:06 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:06.197475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-668d4ce4-4677-4ec2-9bbe-dc431ceae716 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:06.198071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-668d4ce4-4677-4ec2-9bbe-dc431ceae716 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:06.203717 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-668d4ce4-4677-4ec2-9bbe-dc431ceae716 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:06.204382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-668d4ce4-4677-4ec2-9bbe-dc431ceae716 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:06.205336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-668d4ce4-4677-4ec2-9bbe-dc431ceae716 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:06.205973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-668d4ce4-4677-4ec2-9bbe-dc431ceae716 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:06.265779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-668d4ce4-4677-4ec2-9bbe-dc431ceae716 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:06.274680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-668d4ce4-4677-4ec2-9bbe-dc431ceae716 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:06.276210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-668d4ce4-4677-4ec2-9bbe-dc431ceae716 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:06.277357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-668d4ce4-4677-4ec2-9bbe-dc431ceae716 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.083403 Jun 10 13:28:06.277907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1147/2061] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:06 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:07.210262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-51bac512-529b-42a5-81f8-0cb7beeb1a2f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:07.210907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-51bac512-529b-42a5-81f8-0cb7beeb1a2f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:07.213264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:28:07.218229 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-51bac512-529b-42a5-81f8-0cb7beeb1a2f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:07.218816 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-51bac512-529b-42a5-81f8-0cb7beeb1a2f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:07.219621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-51bac512-529b-42a5-81f8-0cb7beeb1a2f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:07.220055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-51bac512-529b-42a5-81f8-0cb7beeb1a2f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:07.352393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-51bac512-529b-42a5-81f8-0cb7beeb1a2f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:07.358437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-51bac512-529b-42a5-81f8-0cb7beeb1a2f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.152025 Jun 10 13:28:07.358988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1148/2062] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:07 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:07.363597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-619f53ca-551b-4324-95fa-5b63fe7d7214 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:07.364299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-619f53ca-551b-4324-95fa-5b63fe7d7214 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:07.369816 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-619f53ca-551b-4324-95fa-5b63fe7d7214 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:07.370530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-619f53ca-551b-4324-95fa-5b63fe7d7214 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:07.370988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-619f53ca-551b-4324-95fa-5b63fe7d7214 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:07.371414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-619f53ca-551b-4324-95fa-5b63fe7d7214 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:07.435636 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-619f53ca-551b-4324-95fa-5b63fe7d7214 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:07.444601 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-619f53ca-551b-4324-95fa-5b63fe7d7214 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:07.446131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-619f53ca-551b-4324-95fa-5b63fe7d7214 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:07.447271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-619f53ca-551b-4324-95fa-5b63fe7d7214 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087372 Jun 10 13:28:07.447834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1149/2063] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:07 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:08.375383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cb6f4649-5b8f-4ab6-8f96-67c000689e45 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:08.376524 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cb6f4649-5b8f-4ab6-8f96-67c000689e45 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:08.383202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb6f4649-5b8f-4ab6-8f96-67c000689e45 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:08.383652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb6f4649-5b8f-4ab6-8f96-67c000689e45 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:08.384410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb6f4649-5b8f-4ab6-8f96-67c000689e45 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:08.384874 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb6f4649-5b8f-4ab6-8f96-67c000689e45 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:08.506950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cb6f4649-5b8f-4ab6-8f96-67c000689e45 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:08.514914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cb6f4649-5b8f-4ab6-8f96-67c000689e45 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143344 Jun 10 13:28:08.515477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1150/2064] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:08 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:08.520468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b2d1182c-e459-4ea3-8773-5630f4ac60db tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:08.521078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b2d1182c-e459-4ea3-8773-5630f4ac60db tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:08.526602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b2d1182c-e459-4ea3-8773-5630f4ac60db tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:08.527049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b2d1182c-e459-4ea3-8773-5630f4ac60db tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:08.527795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b2d1182c-e459-4ea3-8773-5630f4ac60db tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:08.528223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b2d1182c-e459-4ea3-8773-5630f4ac60db tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:08.586504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b2d1182c-e459-4ea3-8773-5630f4ac60db tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:08.594929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b2d1182c-e459-4ea3-8773-5630f4ac60db tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:08.596406 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b2d1182c-e459-4ea3-8773-5630f4ac60db tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:08.597553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b2d1182c-e459-4ea3-8773-5630f4ac60db tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.081175 Jun 10 13:28:08.598106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1151/2065] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:08 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:09.533228 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-906c3c85-1012-446f-b919-ed88752b058a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:09.533850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-906c3c85-1012-446f-b919-ed88752b058a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:09.540429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-906c3c85-1012-446f-b919-ed88752b058a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:09.540903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-906c3c85-1012-446f-b919-ed88752b058a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:09.541648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-906c3c85-1012-446f-b919-ed88752b058a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:09.542076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-906c3c85-1012-446f-b919-ed88752b058a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:09.669157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-906c3c85-1012-446f-b919-ed88752b058a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:09.677120 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-906c3c85-1012-446f-b919-ed88752b058a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148257 Jun 10 13:28:09.677614 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1152/2066] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:09 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:09.681919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4b05a6fc-f024-4a93-846a-bec28132a6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:09.682515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4b05a6fc-f024-4a93-846a-bec28132a6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:09.687997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b05a6fc-f024-4a93-846a-bec28132a6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:09.689008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b05a6fc-f024-4a93-846a-bec28132a6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:09.689762 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b05a6fc-f024-4a93-846a-bec28132a6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:09.690194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b05a6fc-f024-4a93-846a-bec28132a6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:09.749602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4b05a6fc-f024-4a93-846a-bec28132a6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:09.758453 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4b05a6fc-f024-4a93-846a-bec28132a6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:09.759951 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4b05a6fc-f024-4a93-846a-bec28132a6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:09.761116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4b05a6fc-f024-4a93-846a-bec28132a6ff tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.082561 Jun 10 13:28:09.761683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1153/2067] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:09 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:10.694149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c305b306-a6e5-42f6-a3a4-206221d40ca5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:10.694761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c305b306-a6e5-42f6-a3a4-206221d40ca5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:10.701241 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c305b306-a6e5-42f6-a3a4-206221d40ca5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:10.701688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c305b306-a6e5-42f6-a3a4-206221d40ca5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:10.702443 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c305b306-a6e5-42f6-a3a4-206221d40ca5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:10.702874 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c305b306-a6e5-42f6-a3a4-206221d40ca5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:10.826052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c305b306-a6e5-42f6-a3a4-206221d40ca5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:10.834640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c305b306-a6e5-42f6-a3a4-206221d40ca5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144329 Jun 10 13:28:10.835209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1154/2068] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:10 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:10.839835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-407d1feb-6678-4eaf-ad21-5bdc56422508 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:10.840437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-407d1feb-6678-4eaf-ad21-5bdc56422508 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:10.846634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-407d1feb-6678-4eaf-ad21-5bdc56422508 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:10.847080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-407d1feb-6678-4eaf-ad21-5bdc56422508 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:10.847835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-407d1feb-6678-4eaf-ad21-5bdc56422508 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:10.848266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-407d1feb-6678-4eaf-ad21-5bdc56422508 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:10.909461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-407d1feb-6678-4eaf-ad21-5bdc56422508 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:10.917715 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-407d1feb-6678-4eaf-ad21-5bdc56422508 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:10.919210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-407d1feb-6678-4eaf-ad21-5bdc56422508 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:10.920345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-407d1feb-6678-4eaf-ad21-5bdc56422508 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084228 Jun 10 13:28:10.920934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1155/2069] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:10 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:11.851465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-432599a0-2f2c-4fc0-90af-9276042a6290 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:11.852143 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-432599a0-2f2c-4fc0-90af-9276042a6290 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:11.858535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-432599a0-2f2c-4fc0-90af-9276042a6290 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:11.858974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-432599a0-2f2c-4fc0-90af-9276042a6290 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:11.859722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-432599a0-2f2c-4fc0-90af-9276042a6290 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:11.860203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-432599a0-2f2c-4fc0-90af-9276042a6290 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:11.986101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-432599a0-2f2c-4fc0-90af-9276042a6290 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:11.994165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-432599a0-2f2c-4fc0-90af-9276042a6290 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146538 Jun 10 13:28:11.994734 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1156/2070] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:11 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:11.999111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-714757c6-6e4b-4ea5-9185-784f6a7084e1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:11.999703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-714757c6-6e4b-4ea5-9185-784f6a7084e1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:12.005867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-714757c6-6e4b-4ea5-9185-784f6a7084e1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:12.006308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-714757c6-6e4b-4ea5-9185-784f6a7084e1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:12.007055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-714757c6-6e4b-4ea5-9185-784f6a7084e1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:12.007484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-714757c6-6e4b-4ea5-9185-784f6a7084e1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:12.069021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-714757c6-6e4b-4ea5-9185-784f6a7084e1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:12.077263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-714757c6-6e4b-4ea5-9185-784f6a7084e1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:12.079416 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-714757c6-6e4b-4ea5-9185-784f6a7084e1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:12.080538 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-714757c6-6e4b-4ea5-9185-784f6a7084e1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.084891 Jun 10 13:28:12.081135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1157/2071] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:11 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:13.011692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1541c64a-49ff-47de-8396-3860d7f93984 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:13.012303 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1541c64a-49ff-47de-8396-3860d7f93984 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:13.019346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1541c64a-49ff-47de-8396-3860d7f93984 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:13.019795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1541c64a-49ff-47de-8396-3860d7f93984 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:13.020549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1541c64a-49ff-47de-8396-3860d7f93984 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:13.021017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1541c64a-49ff-47de-8396-3860d7f93984 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:13.148144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1541c64a-49ff-47de-8396-3860d7f93984 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:13.156078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1541c64a-49ff-47de-8396-3860d7f93984 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148298 Jun 10 13:28:13.156690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1158/2072] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:13 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:13.161686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dd4684d4-b0b2-4289-b85f-bf3cb5c51733 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:13.162244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dd4684d4-b0b2-4289-b85f-bf3cb5c51733 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:13.167785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd4684d4-b0b2-4289-b85f-bf3cb5c51733 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:13.168236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd4684d4-b0b2-4289-b85f-bf3cb5c51733 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:13.169085 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd4684d4-b0b2-4289-b85f-bf3cb5c51733 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:13.172945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd4684d4-b0b2-4289-b85f-bf3cb5c51733 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:13.228381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dd4684d4-b0b2-4289-b85f-bf3cb5c51733 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:13.237246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dd4684d4-b0b2-4289-b85f-bf3cb5c51733 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:13.238739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dd4684d4-b0b2-4289-b85f-bf3cb5c51733 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:13.239871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dd4684d4-b0b2-4289-b85f-bf3cb5c51733 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.081736 Jun 10 13:28:13.240436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1159/2073] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:13 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:14.173565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-07c78df1-a16b-4465-b44e-8f8cf08a1714 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:14.174235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-07c78df1-a16b-4465-b44e-8f8cf08a1714 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:14.180713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07c78df1-a16b-4465-b44e-8f8cf08a1714 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:14.181172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07c78df1-a16b-4465-b44e-8f8cf08a1714 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:14.181932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07c78df1-a16b-4465-b44e-8f8cf08a1714 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:14.182366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07c78df1-a16b-4465-b44e-8f8cf08a1714 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:14.310191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-07c78df1-a16b-4465-b44e-8f8cf08a1714 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:14.318862 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-07c78df1-a16b-4465-b44e-8f8cf08a1714 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149146 Jun 10 13:28:14.319439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1160/2074] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:14 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:14.324322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b918c048-c8ef-4134-b2ca-b275f85cea0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:14.324958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b918c048-c8ef-4134-b2ca-b275f85cea0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:14.331525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b918c048-c8ef-4134-b2ca-b275f85cea0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:14.331974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b918c048-c8ef-4134-b2ca-b275f85cea0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:14.332737 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b918c048-c8ef-4134-b2ca-b275f85cea0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:14.333203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b918c048-c8ef-4134-b2ca-b275f85cea0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:14.396105 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b918c048-c8ef-4134-b2ca-b275f85cea0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:14.404548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b918c048-c8ef-4134-b2ca-b275f85cea0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:14.406098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b918c048-c8ef-4134-b2ca-b275f85cea0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:14.407828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b918c048-c8ef-4134-b2ca-b275f85cea0b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.087437 Jun 10 13:28:14.408391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1161/2075] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:14 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:15.335748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-51559d45-f3e8-4a97-9b2e-44001211ca6d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:15.336358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-51559d45-f3e8-4a97-9b2e-44001211ca6d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:15.343134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-51559d45-f3e8-4a97-9b2e-44001211ca6d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:15.343582 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-51559d45-f3e8-4a97-9b2e-44001211ca6d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:15.345005 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-51559d45-f3e8-4a97-9b2e-44001211ca6d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:15.345442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-51559d45-f3e8-4a97-9b2e-44001211ca6d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:15.468268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-51559d45-f3e8-4a97-9b2e-44001211ca6d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:15.476718 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-51559d45-f3e8-4a97-9b2e-44001211ca6d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144828 Jun 10 13:28:15.477305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1162/2076] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:15 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:15.481602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fce88520-2a40-4a11-aa05-43699921b08a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:15.482201 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fce88520-2a40-4a11-aa05-43699921b08a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:15.487765 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fce88520-2a40-4a11-aa05-43699921b08a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:15.488210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fce88520-2a40-4a11-aa05-43699921b08a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:15.488989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fce88520-2a40-4a11-aa05-43699921b08a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:15.489419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fce88520-2a40-4a11-aa05-43699921b08a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:15.493293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:28:15.548346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fce88520-2a40-4a11-aa05-43699921b08a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:15.556505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fce88520-2a40-4a11-aa05-43699921b08a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:15.558023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fce88520-2a40-4a11-aa05-43699921b08a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:15.559137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fce88520-2a40-4a11-aa05-43699921b08a tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.080932 Jun 10 13:28:15.559693 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1163/2077] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:15 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:16.494816 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-15ba0526-a71f-48de-9147-727601d62213 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:16.495437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-15ba0526-a71f-48de-9147-727601d62213 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:16.501882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15ba0526-a71f-48de-9147-727601d62213 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:16.502324 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15ba0526-a71f-48de-9147-727601d62213 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:16.503065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15ba0526-a71f-48de-9147-727601d62213 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:16.503492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-15ba0526-a71f-48de-9147-727601d62213 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:16.628866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-15ba0526-a71f-48de-9147-727601d62213 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:16.636887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-15ba0526-a71f-48de-9147-727601d62213 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145879 Jun 10 13:28:16.647999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1164/2078] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:16 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:16.647999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1b9a6ba1-67a7-4d21-8186-394bcf0e0628 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:16.647999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1b9a6ba1-67a7-4d21-8186-394bcf0e0628 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:16.647999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b9a6ba1-67a7-4d21-8186-394bcf0e0628 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:16.648394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b9a6ba1-67a7-4d21-8186-394bcf0e0628 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:16.648928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b9a6ba1-67a7-4d21-8186-394bcf0e0628 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:16.649356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b9a6ba1-67a7-4d21-8186-394bcf0e0628 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:16.711741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1b9a6ba1-67a7-4d21-8186-394bcf0e0628 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:16.721055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1b9a6ba1-67a7-4d21-8186-394bcf0e0628 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:16.722610 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1b9a6ba1-67a7-4d21-8186-394bcf0e0628 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:16.723747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1b9a6ba1-67a7-4d21-8186-394bcf0e0628 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 200 len: 1444 microversion: 2.20 time: 0.085396 Jun 10 13:28:16.724309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1165/2079] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:16 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:16.742541 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-068acf01-c5a9-473a-8c8b-508b30f3b906 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '.version_select at 0x7f9309979ca0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:16.816750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-068acf01-c5a9-473a-8c8b-508b30f3b906 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-966788633" status: 204 len: 0 microversion: 2.20 time: 0.077840 Jun 10 13:28:16.817355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1166/2080] 10.222.0.25 () {60 vars in 1258 bytes} [Fri Jun 10 13:28:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-966788633 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Jun 10 13:28:17.675339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-224062f6-37be-4037-81f6-9b497be8c793 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:17.675339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-224062f6-37be-4037-81f6-9b497be8c793 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:17.678775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-224062f6-37be-4037-81f6-9b497be8c793 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:17.679220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-224062f6-37be-4037-81f6-9b497be8c793 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:17.679985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-224062f6-37be-4037-81f6-9b497be8c793 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:17.680416 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-224062f6-37be-4037-81f6-9b497be8c793 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:17.814863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-224062f6-37be-4037-81f6-9b497be8c793 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:17.822714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-224062f6-37be-4037-81f6-9b497be8c793 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.163311 Jun 10 13:28:17.823274 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1167/2081] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:17 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:28:17.827980 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cb47f1b1-e963-48bf-aa5a-2a8350607ed6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Action: 'create', calling method: .version_select at 0x7f93052dab80>, body: {"keypair": {"name": "tempest-keypair-224439963", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC2iQSelinn4fJuxYcOodvAauR4++Q3LcgMNFPNlxHXK+CTpST7AB5Z1k2psabxSrzWBCd8duqwGtoHTCS+6ymu/9kFSCkjkdjgD5GXW3qFLStDyyKZcM/lX3l4ks1AtMQ=="}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:28:17.841221 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-cb47f1b1-e963-48bf-aa5a-2a8350607ed6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Getting quotas for project 28f8244decff453e98e2d372a97a563c. Resources: {'key_pairs'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:28:17.845014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-cb47f1b1-e963-48bf-aa5a-2a8350607ed6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Getting quotas for user b1d1712ea523498f92f258383863b906 and project 28f8244decff453e98e2d372a97a563c. Resources: {'key_pairs'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:28:17.937772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cb47f1b1-e963-48bf-aa5a-2a8350607ed6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.113363 Jun 10 13:28:17.938263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1168/2082] 10.222.0.25 () {62 vars in 1199 bytes} [Fri Jun 10 13:28:17 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 114 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jun 10 13:28:18.840423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f0b6fe03-47e8-4d04-b8ad-5c3d3fffa3f5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:18.841098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f0b6fe03-47e8-4d04-b8ad-5c3d3fffa3f5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:18.848975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0b6fe03-47e8-4d04-b8ad-5c3d3fffa3f5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:18.851818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0b6fe03-47e8-4d04-b8ad-5c3d3fffa3f5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:18.851818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0b6fe03-47e8-4d04-b8ad-5c3d3fffa3f5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:18.851818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0b6fe03-47e8-4d04-b8ad-5c3d3fffa3f5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:18.981773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f0b6fe03-47e8-4d04-b8ad-5c3d3fffa3f5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:18.990588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f0b6fe03-47e8-4d04-b8ad-5c3d3fffa3f5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.154863 Jun 10 13:28:18.991315 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1169/2083] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:18 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:20.012232 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-74d8531e-aa89-4943-8cb6-79a284f7a00c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:20.013560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-74d8531e-aa89-4943-8cb6-79a284f7a00c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:20.025076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74d8531e-aa89-4943-8cb6-79a284f7a00c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:20.026024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74d8531e-aa89-4943-8cb6-79a284f7a00c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:20.027307 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74d8531e-aa89-4943-8cb6-79a284f7a00c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:20.028037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74d8531e-aa89-4943-8cb6-79a284f7a00c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:20.168145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-74d8531e-aa89-4943-8cb6-79a284f7a00c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:20.176921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-74d8531e-aa89-4943-8cb6-79a284f7a00c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.172084 Jun 10 13:28:20.177429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1170/2084] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:20 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:28:20.430138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-51596863", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a83e307f-ca79-4dcf-bb7e-cc270b1f93af"}], "security_groups": [{"name": "tempest-securitygroup--77327039"}], "key_name": "tempest-keypair-224439963", "user_data": "}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:28:20.531608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.network.neutron [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] validate_networks() for [('a83e307f-ca79-4dcf-bb7e-cc270b1f93af', None, None, None, None, None)] {{(pid=118092) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:28:20.958305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Flavor limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:28:20.958748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Image limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:28:20.959229 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Flavor pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:28:20.959637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Image pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:28:20.960072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:28:20.969164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:20.970219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:20.971009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:20.971442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:20.988452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Getting quotas for project 28f8244decff453e98e2d372a97a563c. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:28:20.992391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Getting quotas for user b1d1712ea523498f92f258383863b906 and project 28f8244decff453e98e2d372a97a563c. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:28:21.003231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Going to run 1 instances... {{(pid=118092) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:28:21.005482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118092) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:28:21.161513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1f922290-d722-4b74-bf37-1a9b94d6689d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.20 time: 0.735656 Jun 10 13:28:21.167949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1171/2085] 10.222.0.25 () {62 vars in 1187 bytes} [Fri Jun 10 13:28:20 2022] POST /compute/v2.1/servers => generated 400 bytes in 743 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jun 10 13:28:21.173363 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9e41dfce-2221-4bcb-97f7-d3fa13f45ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:21.173983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9e41dfce-2221-4bcb-97f7-d3fa13f45ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:21.225178 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9e41dfce-2221-4bcb-97f7-d3fa13f45ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:21.232293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9e41dfce-2221-4bcb-97f7-d3fa13f45ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:21.233813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9e41dfce-2221-4bcb-97f7-d3fa13f45ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:21.234909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9e41dfce-2221-4bcb-97f7-d3fa13f45ab7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1279 microversion: 2.20 time: 0.065946 Jun 10 13:28:21.236047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1172/2086] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:21 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1279 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:21.240471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0f386cbd-f7d0-4273-8256-0472b6d2316c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:21.241093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0f386cbd-f7d0-4273-8256-0472b6d2316c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:21.250361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f386cbd-f7d0-4273-8256-0472b6d2316c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:21.250851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f386cbd-f7d0-4273-8256-0472b6d2316c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:21.251622 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f386cbd-f7d0-4273-8256-0472b6d2316c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:21.252091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f386cbd-f7d0-4273-8256-0472b6d2316c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:21.379124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0f386cbd-f7d0-4273-8256-0472b6d2316c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:21.384855 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0f386cbd-f7d0-4273-8256-0472b6d2316c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147842 Jun 10 13:28:21.385431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1173/2087] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:21 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:28:22.255781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4d582da5-ce38-4f8d-8949-6accdfd82037 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:22.256490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4d582da5-ce38-4f8d-8949-6accdfd82037 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:22.310787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4d582da5-ce38-4f8d-8949-6accdfd82037 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:22.318826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4d582da5-ce38-4f8d-8949-6accdfd82037 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:22.320333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4d582da5-ce38-4f8d-8949-6accdfd82037 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:22.321489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4d582da5-ce38-4f8d-8949-6accdfd82037 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1279 microversion: 2.20 time: 0.070684 Jun 10 13:28:22.322207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1174/2088] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:22 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1279 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:28:22.402337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-839b5ddc-2896-45aa-bcb3-4cdd41468eb5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:22.402947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-839b5ddc-2896-45aa-bcb3-4cdd41468eb5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:22.410277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-839b5ddc-2896-45aa-bcb3-4cdd41468eb5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:22.410722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-839b5ddc-2896-45aa-bcb3-4cdd41468eb5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:22.411475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-839b5ddc-2896-45aa-bcb3-4cdd41468eb5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:22.411911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-839b5ddc-2896-45aa-bcb3-4cdd41468eb5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:22.533434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-839b5ddc-2896-45aa-bcb3-4cdd41468eb5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:22.541255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-839b5ddc-2896-45aa-bcb3-4cdd41468eb5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.142717 Jun 10 13:28:22.541816 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1175/2089] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:22 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:23.339495 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0f4dd0af-5ed8-4a37-85a6-e3a332fd5140 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:23.340104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0f4dd0af-5ed8-4a37-85a6-e3a332fd5140 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:23.391016 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0f4dd0af-5ed8-4a37-85a6-e3a332fd5140 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:23.398175 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0f4dd0af-5ed8-4a37-85a6-e3a332fd5140 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:23.399671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0f4dd0af-5ed8-4a37-85a6-e3a332fd5140 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:23.400795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0f4dd0af-5ed8-4a37-85a6-e3a332fd5140 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1279 microversion: 2.20 time: 0.065473 Jun 10 13:28:23.401358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1176/2090] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:23 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1279 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:28:23.559087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8accd99f-af53-4254-af76-3bace3bc86ba tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:23.559697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8accd99f-af53-4254-af76-3bace3bc86ba tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:23.565750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8accd99f-af53-4254-af76-3bace3bc86ba tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:23.566200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8accd99f-af53-4254-af76-3bace3bc86ba tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:23.566944 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8accd99f-af53-4254-af76-3bace3bc86ba tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:23.567394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8accd99f-af53-4254-af76-3bace3bc86ba tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:23.694984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8accd99f-af53-4254-af76-3bace3bc86ba tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:23.702833 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8accd99f-af53-4254-af76-3bace3bc86ba tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147624 Jun 10 13:28:23.703401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1177/2091] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:23 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:24.419026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-10d7a0f4-a99c-4a9c-8fae-7723f1be5053 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:24.419770 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-10d7a0f4-a99c-4a9c-8fae-7723f1be5053 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:24.471254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10d7a0f4-a99c-4a9c-8fae-7723f1be5053 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:24.478451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10d7a0f4-a99c-4a9c-8fae-7723f1be5053 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:24.479945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10d7a0f4-a99c-4a9c-8fae-7723f1be5053 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:24.481066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-10d7a0f4-a99c-4a9c-8fae-7723f1be5053 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1279 microversion: 2.20 time: 0.066030 Jun 10 13:28:24.481624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1178/2092] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:24 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1279 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:28:24.720217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dac9e978-e2b0-4d23-91c5-83516375fa48 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:24.720851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dac9e978-e2b0-4d23-91c5-83516375fa48 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:24.726847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dac9e978-e2b0-4d23-91c5-83516375fa48 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:24.727294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dac9e978-e2b0-4d23-91c5-83516375fa48 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:24.728044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dac9e978-e2b0-4d23-91c5-83516375fa48 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:24.728473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dac9e978-e2b0-4d23-91c5-83516375fa48 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:24.853870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dac9e978-e2b0-4d23-91c5-83516375fa48 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:24.861890 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dac9e978-e2b0-4d23-91c5-83516375fa48 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145485 Jun 10 13:28:24.862454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1179/2093] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:24 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:28:25.498753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d71cc15a-b539-4729-bb5b-b339299156d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:25.499362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d71cc15a-b539-4729-bb5b-b339299156d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:25.501744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=95,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:28:25.506479 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d71cc15a-b539-4729-bb5b-b339299156d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:25.506920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d71cc15a-b539-4729-bb5b-b339299156d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:25.507663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d71cc15a-b539-4729-bb5b-b339299156d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:25.508092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d71cc15a-b539-4729-bb5b-b339299156d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:25.570375 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d71cc15a-b539-4729-bb5b-b339299156d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:25.578684 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d71cc15a-b539-4729-bb5b-b339299156d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:25.580182 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d71cc15a-b539-4729-bb5b-b339299156d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:25.581314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d71cc15a-b539-4729-bb5b-b339299156d8 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1283 microversion: 2.20 time: 0.086512 Jun 10 13:28:25.581865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1180/2094] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:25 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1283 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:28:25.879319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9cfe7d9f-ac62-4150-8d05-c813b8fd4436 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:25.879857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9cfe7d9f-ac62-4150-8d05-c813b8fd4436 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:25.885691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cfe7d9f-ac62-4150-8d05-c813b8fd4436 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:25.886143 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cfe7d9f-ac62-4150-8d05-c813b8fd4436 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:25.886903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cfe7d9f-ac62-4150-8d05-c813b8fd4436 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:25.887343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cfe7d9f-ac62-4150-8d05-c813b8fd4436 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:26.010714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9cfe7d9f-ac62-4150-8d05-c813b8fd4436 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:26.018995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9cfe7d9f-ac62-4150-8d05-c813b8fd4436 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144135 Jun 10 13:28:26.019565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1181/2095] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:25 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:26.598551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fb69a30f-fea9-44b6-b12f-574f8912c521 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:26.599205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fb69a30f-fea9-44b6-b12f-574f8912c521 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:26.605794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb69a30f-fea9-44b6-b12f-574f8912c521 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:26.606237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb69a30f-fea9-44b6-b12f-574f8912c521 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:26.606982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb69a30f-fea9-44b6-b12f-574f8912c521 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:26.607407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb69a30f-fea9-44b6-b12f-574f8912c521 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:26.670947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fb69a30f-fea9-44b6-b12f-574f8912c521 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:26.679612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fb69a30f-fea9-44b6-b12f-574f8912c521 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:26.681205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fb69a30f-fea9-44b6-b12f-574f8912c521 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:26.682304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fb69a30f-fea9-44b6-b12f-574f8912c521 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1275 microversion: 2.20 time: 0.087651 Jun 10 13:28:26.682858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1182/2096] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:26 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1275 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:28:27.036432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c4eb3442-b5f4-45e3-824a-2b4f28749b19 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:27.037060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c4eb3442-b5f4-45e3-824a-2b4f28749b19 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:27.043089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c4eb3442-b5f4-45e3-824a-2b4f28749b19 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:27.043536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c4eb3442-b5f4-45e3-824a-2b4f28749b19 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:27.044289 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c4eb3442-b5f4-45e3-824a-2b4f28749b19 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:27.044723 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c4eb3442-b5f4-45e3-824a-2b4f28749b19 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:27.174583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c4eb3442-b5f4-45e3-824a-2b4f28749b19 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:27.182581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c4eb3442-b5f4-45e3-824a-2b4f28749b19 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150019 Jun 10 13:28:27.183345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1183/2097] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:27 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:27.701260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c3158d66-4d13-42cf-ac0b-3a65b4fc1d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:27.701904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c3158d66-4d13-42cf-ac0b-3a65b4fc1d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:27.709953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3158d66-4d13-42cf-ac0b-3a65b4fc1d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:27.710719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3158d66-4d13-42cf-ac0b-3a65b4fc1d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:27.712041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3158d66-4d13-42cf-ac0b-3a65b4fc1d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:27.712567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3158d66-4d13-42cf-ac0b-3a65b4fc1d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:27.779987 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c3158d66-4d13-42cf-ac0b-3a65b4fc1d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:27.787938 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c3158d66-4d13-42cf-ac0b-3a65b4fc1d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:27.789459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c3158d66-4d13-42cf-ac0b-3a65b4fc1d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:27.790587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c3158d66-4d13-42cf-ac0b-3a65b4fc1d26 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1337 microversion: 2.20 time: 0.093284 Jun 10 13:28:27.800909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1184/2098] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:27 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1337 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:28.199623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d052b653-b069-4f8d-87c5-cb77a6fa9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:28.200230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d052b653-b069-4f8d-87c5-cb77a6fa9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:28.206568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d052b653-b069-4f8d-87c5-cb77a6fa9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:28.207016 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d052b653-b069-4f8d-87c5-cb77a6fa9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:28.207794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d052b653-b069-4f8d-87c5-cb77a6fa9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:28.208217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d052b653-b069-4f8d-87c5-cb77a6fa9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:28.342631 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d052b653-b069-4f8d-87c5-cb77a6fa9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:28.350982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d052b653-b069-4f8d-87c5-cb77a6fa9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.155349 Jun 10 13:28:28.351805 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1185/2099] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:28 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:28.807580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-89edac87-017e-4211-b8c1-b1ab6ca45ad7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:28.808263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-89edac87-017e-4211-b8c1-b1ab6ca45ad7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:28.815902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-89edac87-017e-4211-b8c1-b1ab6ca45ad7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:28.816661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-89edac87-017e-4211-b8c1-b1ab6ca45ad7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:28.817551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-89edac87-017e-4211-b8c1-b1ab6ca45ad7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:28.818047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-89edac87-017e-4211-b8c1-b1ab6ca45ad7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:28.837808 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:28:28.953746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-89edac87-017e-4211-b8c1-b1ab6ca45ad7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:28.961520 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-89edac87-017e-4211-b8c1-b1ab6ca45ad7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:28.963022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-89edac87-017e-4211-b8c1-b1ab6ca45ad7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:28.964399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-89edac87-017e-4211-b8c1-b1ab6ca45ad7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1403 microversion: 2.20 time: 0.160719 Jun 10 13:28:28.964991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1186/2100] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:28 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1403 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jun 10 13:28:29.109337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-4821e484-6570-4a74-a4b9-81091668e41c req-7234b644-666d-42ab-8b3e-77c543b8a065 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a99c1a73-3f56-4c71-a4fd-67988bbb7464", "tag": "a5d10747-5fee-4e44-ae6e-53bfbd5134eb"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:28:29.118172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-4821e484-6570-4a74-a4b9-81091668e41c req-7234b644-666d-42ab-8b3e-77c543b8a065 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:29.118611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-4821e484-6570-4a74-a4b9-81091668e41c req-7234b644-666d-42ab-8b3e-77c543b8a065 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:29.130787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-4821e484-6570-4a74-a4b9-81091668e41c req-7234b644-666d-42ab-8b3e-77c543b8a065 service nova] Creating event network-changed:a5d10747-5fee-4e44-ae6e-53bfbd5134eb for instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 on hv-cinder-maste Jun 10 13:28:29.135903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-4821e484-6570-4a74-a4b9-81091668e41c req-7234b644-666d-42ab-8b3e-77c543b8a065 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.121722 Jun 10 13:28:29.136521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1187/2101] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:28:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:28:29.369254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3767b785-026b-47ce-ac27-6367a6818cc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:29.369951 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3767b785-026b-47ce-ac27-6367a6818cc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:29.376153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3767b785-026b-47ce-ac27-6367a6818cc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:29.376602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3767b785-026b-47ce-ac27-6367a6818cc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:29.377395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3767b785-026b-47ce-ac27-6367a6818cc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:29.377862 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3767b785-026b-47ce-ac27-6367a6818cc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:29.505856 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3767b785-026b-47ce-ac27-6367a6818cc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:29.513745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3767b785-026b-47ce-ac27-6367a6818cc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148663 Jun 10 13:28:29.514313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1188/2102] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:29 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:29.982575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-115ba571-8e9a-4b45-b06c-e83f804e02cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:29.983239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-115ba571-8e9a-4b45-b06c-e83f804e02cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:29.990275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-115ba571-8e9a-4b45-b06c-e83f804e02cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:29.990727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-115ba571-8e9a-4b45-b06c-e83f804e02cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:29.991496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-115ba571-8e9a-4b45-b06c-e83f804e02cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:29.991919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-115ba571-8e9a-4b45-b06c-e83f804e02cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:30.118055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-115ba571-8e9a-4b45-b06c-e83f804e02cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:30.125915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-115ba571-8e9a-4b45-b06c-e83f804e02cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:30.127410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-115ba571-8e9a-4b45-b06c-e83f804e02cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:30.128505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-115ba571-8e9a-4b45-b06c-e83f804e02cc tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1403 microversion: 2.20 time: 0.149957 Jun 10 13:28:30.129111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1189/2103] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:29 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1403 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:28:30.530326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dd922eb2-28c3-4450-b853-6dd6ea50fb3e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:30.530942 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dd922eb2-28c3-4450-b853-6dd6ea50fb3e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:30.537073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd922eb2-28c3-4450-b853-6dd6ea50fb3e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:30.537506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd922eb2-28c3-4450-b853-6dd6ea50fb3e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:30.538254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd922eb2-28c3-4450-b853-6dd6ea50fb3e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:30.538685 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd922eb2-28c3-4450-b853-6dd6ea50fb3e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:30.669086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dd922eb2-28c3-4450-b853-6dd6ea50fb3e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:30.677815 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dd922eb2-28c3-4450-b853-6dd6ea50fb3e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.151351 Jun 10 13:28:30.678391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1190/2104] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:30 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:31.147628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d4f70e63-0a14-41c9-af91-6d614c99fed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:31.148236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d4f70e63-0a14-41c9-af91-6d614c99fed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:31.150419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:28:31.155470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4f70e63-0a14-41c9-af91-6d614c99fed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:31.155921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4f70e63-0a14-41c9-af91-6d614c99fed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:31.157326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4f70e63-0a14-41c9-af91-6d614c99fed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:31.157772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d4f70e63-0a14-41c9-af91-6d614c99fed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:31.283345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d4f70e63-0a14-41c9-af91-6d614c99fed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:31.291148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d4f70e63-0a14-41c9-af91-6d614c99fed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:31.292643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d4f70e63-0a14-41c9-af91-6d614c99fed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:31.293781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d4f70e63-0a14-41c9-af91-6d614c99fed1 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1403 microversion: 2.20 time: 0.150808 Jun 10 13:28:31.294340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1191/2105] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:31 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1403 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:28:31.694695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-64bbb9c4-0549-4cd7-87d7-c809649d1643 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:31.695304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-64bbb9c4-0549-4cd7-87d7-c809649d1643 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:31.701469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-64bbb9c4-0549-4cd7-87d7-c809649d1643 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:31.701910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-64bbb9c4-0549-4cd7-87d7-c809649d1643 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:31.702659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-64bbb9c4-0549-4cd7-87d7-c809649d1643 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:31.703086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-64bbb9c4-0549-4cd7-87d7-c809649d1643 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:31.831009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-64bbb9c4-0549-4cd7-87d7-c809649d1643 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:31.839048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-64bbb9c4-0549-4cd7-87d7-c809649d1643 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148305 Jun 10 13:28:31.839604 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1192/2106] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:31 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:32.310702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-91f30fad-130d-4675-9729-02efd6cf08af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:32.311305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-91f30fad-130d-4675-9729-02efd6cf08af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:32.317824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91f30fad-130d-4675-9729-02efd6cf08af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:32.322351 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91f30fad-130d-4675-9729-02efd6cf08af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:32.322351 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91f30fad-130d-4675-9729-02efd6cf08af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:32.322351 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91f30fad-130d-4675-9729-02efd6cf08af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:32.448112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-91f30fad-130d-4675-9729-02efd6cf08af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:32.453504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-91f30fad-130d-4675-9729-02efd6cf08af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:32.455023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-91f30fad-130d-4675-9729-02efd6cf08af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:32.456120 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-91f30fad-130d-4675-9729-02efd6cf08af tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1403 microversion: 2.20 time: 0.149352 Jun 10 13:28:32.456671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1193/2107] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:32 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1403 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:28:32.855432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-db9f9dba-cc50-4b44-ae36-5972db87a022 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:32.856041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-db9f9dba-cc50-4b44-ae36-5972db87a022 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:32.862336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-db9f9dba-cc50-4b44-ae36-5972db87a022 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:32.862778 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-db9f9dba-cc50-4b44-ae36-5972db87a022 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:32.863526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-db9f9dba-cc50-4b44-ae36-5972db87a022 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:32.863950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-db9f9dba-cc50-4b44-ae36-5972db87a022 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:32.990964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-db9f9dba-cc50-4b44-ae36-5972db87a022 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:32.998828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-db9f9dba-cc50-4b44-ae36-5972db87a022 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147247 Jun 10 13:28:32.999394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1194/2108] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:32 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:33.474217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-35315b7b-114f-460b-8dda-2b16151d45e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:33.474823 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-35315b7b-114f-460b-8dda-2b16151d45e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:33.481647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35315b7b-114f-460b-8dda-2b16151d45e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:33.482091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35315b7b-114f-460b-8dda-2b16151d45e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:33.482836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35315b7b-114f-460b-8dda-2b16151d45e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:33.483266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35315b7b-114f-460b-8dda-2b16151d45e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:33.607165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-35315b7b-114f-460b-8dda-2b16151d45e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:33.615564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-35315b7b-114f-460b-8dda-2b16151d45e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:33.617078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-35315b7b-114f-460b-8dda-2b16151d45e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:33.618231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-35315b7b-114f-460b-8dda-2b16151d45e5 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1403 microversion: 2.20 time: 0.147943 Jun 10 13:28:33.618981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1195/2109] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:33 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1403 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:28:34.016834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-648d8939-18c6-4d5f-8ca0-bb05e5748e3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:34.017442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-648d8939-18c6-4d5f-8ca0-bb05e5748e3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:34.023449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-648d8939-18c6-4d5f-8ca0-bb05e5748e3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:34.023893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-648d8939-18c6-4d5f-8ca0-bb05e5748e3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:34.024644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-648d8939-18c6-4d5f-8ca0-bb05e5748e3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:34.025101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-648d8939-18c6-4d5f-8ca0-bb05e5748e3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:34.146073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-648d8939-18c6-4d5f-8ca0-bb05e5748e3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:34.153933 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-648d8939-18c6-4d5f-8ca0-bb05e5748e3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.141731 Jun 10 13:28:34.154494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1196/2110] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:34 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:34.636463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0fd9a3ea-beba-41f4-b82d-e86b6cf1c328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:34.637082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0fd9a3ea-beba-41f4-b82d-e86b6cf1c328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:34.638935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=18 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:28:34.644109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fd9a3ea-beba-41f4-b82d-e86b6cf1c328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:34.644551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fd9a3ea-beba-41f4-b82d-e86b6cf1c328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:34.645334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fd9a3ea-beba-41f4-b82d-e86b6cf1c328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:34.645761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fd9a3ea-beba-41f4-b82d-e86b6cf1c328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:34.774655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0fd9a3ea-beba-41f4-b82d-e86b6cf1c328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:34.784902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0fd9a3ea-beba-41f4-b82d-e86b6cf1c328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:34.786441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0fd9a3ea-beba-41f4-b82d-e86b6cf1c328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:34.787552 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0fd9a3ea-beba-41f4-b82d-e86b6cf1c328 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1403 microversion: 2.20 time: 0.155082 Jun 10 13:28:34.788101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1197/2111] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:34 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1403 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:28:35.170745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-baf2756f-2099-442e-8df4-5065264ae33e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:35.171357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-baf2756f-2099-442e-8df4-5065264ae33e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:35.177384 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-baf2756f-2099-442e-8df4-5065264ae33e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:35.177867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-baf2756f-2099-442e-8df4-5065264ae33e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:35.178616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-baf2756f-2099-442e-8df4-5065264ae33e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:35.179043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-baf2756f-2099-442e-8df4-5065264ae33e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:35.312966 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-baf2756f-2099-442e-8df4-5065264ae33e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:35.321134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-baf2756f-2099-442e-8df4-5065264ae33e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.154127 Jun 10 13:28:35.322034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1198/2112] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:35 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:35.417432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-f6b69471-5b06-444e-8770-42777399a4c4 req-6ca70bdf-579d-40dd-ac71-fce34f8b09b9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a99c1a73-3f56-4c71-a4fd-67988bbb7464", "name": "network-vif-plugged", "status": "completed", "tag": "a5d10747-5fee-4e44-ae6e-53bfbd5134eb"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:28:35.425588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-f6b69471-5b06-444e-8770-42777399a4c4 req-6ca70bdf-579d-40dd-ac71-fce34f8b09b9 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:35.426022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-f6b69471-5b06-444e-8770-42777399a4c4 req-6ca70bdf-579d-40dd-ac71-fce34f8b09b9 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:35.436897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-f6b69471-5b06-444e-8770-42777399a4c4 req-6ca70bdf-579d-40dd-ac71-fce34f8b09b9 service nova] Creating event network-vif-plugged:a5d10747-5fee-4e44-ae6e-53bfbd5134eb for instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 on hv-cinder-maste Jun 10 13:28:35.442445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-f6b69471-5b06-444e-8770-42777399a4c4 req-6ca70bdf-579d-40dd-ac71-fce34f8b09b9 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082415 Jun 10 13:28:35.443177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1199/2113] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:28:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:28:35.806774 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-17d17d42-312c-470f-b25a-4ff0d44fec8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:35.807600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-17d17d42-312c-470f-b25a-4ff0d44fec8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:35.813960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17d17d42-312c-470f-b25a-4ff0d44fec8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:35.814409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17d17d42-312c-470f-b25a-4ff0d44fec8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:35.815173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17d17d42-312c-470f-b25a-4ff0d44fec8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:35.815607 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17d17d42-312c-470f-b25a-4ff0d44fec8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:35.945231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-17d17d42-312c-470f-b25a-4ff0d44fec8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:35.953149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-17d17d42-312c-470f-b25a-4ff0d44fec8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:35.955158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-17d17d42-312c-470f-b25a-4ff0d44fec8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:35.956255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-17d17d42-312c-470f-b25a-4ff0d44fec8e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1403 microversion: 2.20 time: 0.154431 Jun 10 13:28:35.956829 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1200/2114] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:35 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1403 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:28:36.339289 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3345bb63-e2a7-4752-9ce5-631fa3441741 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:36.339896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3345bb63-e2a7-4752-9ce5-631fa3441741 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:36.345890 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3345bb63-e2a7-4752-9ce5-631fa3441741 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:36.346334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3345bb63-e2a7-4752-9ce5-631fa3441741 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:36.347082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3345bb63-e2a7-4752-9ce5-631fa3441741 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:36.347511 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3345bb63-e2a7-4752-9ce5-631fa3441741 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:36.469290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3345bb63-e2a7-4752-9ce5-631fa3441741 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:36.477135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3345bb63-e2a7-4752-9ce5-631fa3441741 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.141981 Jun 10 13:28:36.477697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1201/2115] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:36 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:36.975254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c44ea412-e054-47ae-b203-523c06025f32 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:36.975925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c44ea412-e054-47ae-b203-523c06025f32 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:36.982499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c44ea412-e054-47ae-b203-523c06025f32 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:36.982950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c44ea412-e054-47ae-b203-523c06025f32 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:36.983715 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c44ea412-e054-47ae-b203-523c06025f32 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:36.984154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c44ea412-e054-47ae-b203-523c06025f32 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:37.112889 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c44ea412-e054-47ae-b203-523c06025f32 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:37.120972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c44ea412-e054-47ae-b203-523c06025f32 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:37.122577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c44ea412-e054-47ae-b203-523c06025f32 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:37.123691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c44ea412-e054-47ae-b203-523c06025f32 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1403 microversion: 2.20 time: 0.152359 Jun 10 13:28:37.124256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1202/2116] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:36 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1403 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:28:37.494080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-891798e2-7a9f-47d5-b24c-67c84cc7da4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:37.494691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-891798e2-7a9f-47d5-b24c-67c84cc7da4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:37.500696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-891798e2-7a9f-47d5-b24c-67c84cc7da4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:37.501689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-891798e2-7a9f-47d5-b24c-67c84cc7da4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:37.502443 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-891798e2-7a9f-47d5-b24c-67c84cc7da4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:37.502874 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-891798e2-7a9f-47d5-b24c-67c84cc7da4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:37.967783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-891798e2-7a9f-47d5-b24c-67c84cc7da4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:37.975717 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-891798e2-7a9f-47d5-b24c-67c84cc7da4f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.485655 Jun 10 13:28:37.976307 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1203/2117] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:37 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:38.140242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3af67531-0578-4644-aa53-8917f1c345c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:38.140792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3af67531-0578-4644-aa53-8917f1c345c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:38.147071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3af67531-0578-4644-aa53-8917f1c345c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:38.147520 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3af67531-0578-4644-aa53-8917f1c345c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:38.148267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3af67531-0578-4644-aa53-8917f1c345c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:38.148724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3af67531-0578-4644-aa53-8917f1c345c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:38.166711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:28:38.272468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3af67531-0578-4644-aa53-8917f1c345c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:38.283608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3af67531-0578-4644-aa53-8917f1c345c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:38.285128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3af67531-0578-4644-aa53-8917f1c345c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:38.286274 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3af67531-0578-4644-aa53-8917f1c345c6 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1596 microversion: 2.20 time: 0.149842 Jun 10 13:28:38.286823 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1204/2118] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:28:38 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1596 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:28:38.303303 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5de067f9-364d-47e2-b705-fa0e1a26d1d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:38.303790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5de067f9-364d-47e2-b705-fa0e1a26d1d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:38.309663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5de067f9-364d-47e2-b705-fa0e1a26d1d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:38.310111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5de067f9-364d-47e2-b705-fa0e1a26d1d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:38.310861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5de067f9-364d-47e2-b705-fa0e1a26d1d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:38.311294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5de067f9-364d-47e2-b705-fa0e1a26d1d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:38.386968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5de067f9-364d-47e2-b705-fa0e1a26d1d7 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.087109 Jun 10 13:28:38.387564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1205/2119] 10.222.0.25 () {60 vars in 1315 bytes} [Fri Jun 10 13:28:38 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464/os-interface => generated 285 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:28:38.992504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3ed054bf-fefd-4a30-af7c-da0e25435fb1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:38.993150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3ed054bf-fefd-4a30-af7c-da0e25435fb1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:38.999338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3ed054bf-fefd-4a30-af7c-da0e25435fb1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:38.999797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3ed054bf-fefd-4a30-af7c-da0e25435fb1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:39.000567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3ed054bf-fefd-4a30-af7c-da0e25435fb1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:39.001060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3ed054bf-fefd-4a30-af7c-da0e25435fb1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:39.214612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3ed054bf-fefd-4a30-af7c-da0e25435fb1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:39.223404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3ed054bf-fefd-4a30-af7c-da0e25435fb1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.234790 Jun 10 13:28:39.224084 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1206/2120] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:38 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:39.449247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-c1fe41b9-a20b-4d0d-a61f-b74d733b4da5 req-ce991918-9a09-4fd8-8133-6d90edadaa4e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a99c1a73-3f56-4c71-a4fd-67988bbb7464", "tag": "a5d10747-5fee-4e44-ae6e-53bfbd5134eb"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:28:39.457593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-c1fe41b9-a20b-4d0d-a61f-b74d733b4da5 req-ce991918-9a09-4fd8-8133-6d90edadaa4e service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:39.458036 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-c1fe41b9-a20b-4d0d-a61f-b74d733b4da5 req-ce991918-9a09-4fd8-8133-6d90edadaa4e service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:39.468927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-c1fe41b9-a20b-4d0d-a61f-b74d733b4da5 req-ce991918-9a09-4fd8-8133-6d90edadaa4e service nova] Creating event network-changed:a5d10747-5fee-4e44-ae6e-53bfbd5134eb for instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 on hv-cinder-maste Jun 10 13:28:39.474580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-c1fe41b9-a20b-4d0d-a61f-b74d733b4da5 req-ce991918-9a09-4fd8-8133-6d90edadaa4e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.079952 Jun 10 13:28:39.475341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1207/2121] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:28:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:28:40.242353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2dde8c43-86e4-4970-ab37-4c0b466d5870 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:40.243086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2dde8c43-86e4-4970-ab37-4c0b466d5870 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:40.249829 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2dde8c43-86e4-4970-ab37-4c0b466d5870 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:40.250286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2dde8c43-86e4-4970-ab37-4c0b466d5870 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:40.251625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2dde8c43-86e4-4970-ab37-4c0b466d5870 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:40.252068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2dde8c43-86e4-4970-ab37-4c0b466d5870 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:40.387056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2dde8c43-86e4-4970-ab37-4c0b466d5870 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:40.395299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2dde8c43-86e4-4970-ab37-4c0b466d5870 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.157799 Jun 10 13:28:40.395882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1208/2122] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:40 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:41.411921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e6b3357d-671b-4f2c-84bd-96f1043f62ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:41.412539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e6b3357d-671b-4f2c-84bd-96f1043f62ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:41.419076 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e6b3357d-671b-4f2c-84bd-96f1043f62ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:41.419525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e6b3357d-671b-4f2c-84bd-96f1043f62ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:41.420277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e6b3357d-671b-4f2c-84bd-96f1043f62ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:41.420711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e6b3357d-671b-4f2c-84bd-96f1043f62ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:41.552192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e6b3357d-671b-4f2c-84bd-96f1043f62ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:41.560068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e6b3357d-671b-4f2c-84bd-96f1043f62ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.152031 Jun 10 13:28:41.560624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1209/2123] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:41 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:42.576703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-48eee7bd-f216-42be-b571-f6a06107a1bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:42.577351 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-48eee7bd-f216-42be-b571-f6a06107a1bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:42.583772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48eee7bd-f216-42be-b571-f6a06107a1bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:42.584228 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48eee7bd-f216-42be-b571-f6a06107a1bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:42.585025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48eee7bd-f216-42be-b571-f6a06107a1bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:42.585463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-48eee7bd-f216-42be-b571-f6a06107a1bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:42.709728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-48eee7bd-f216-42be-b571-f6a06107a1bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:42.717822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-48eee7bd-f216-42be-b571-f6a06107a1bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144955 Jun 10 13:28:42.718391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1210/2124] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:42 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:43.734270 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f862a511-f469-416a-8963-92b79135bdc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:43.734931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f862a511-f469-416a-8963-92b79135bdc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:43.741876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f862a511-f469-416a-8963-92b79135bdc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:43.742316 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f862a511-f469-416a-8963-92b79135bdc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:43.743061 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f862a511-f469-416a-8963-92b79135bdc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:43.743489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f862a511-f469-416a-8963-92b79135bdc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:43.871441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f862a511-f469-416a-8963-92b79135bdc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:43.879903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f862a511-f469-416a-8963-92b79135bdc5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149529 Jun 10 13:28:43.880469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1211/2125] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:43 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:44.897006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c1408a9f-8983-43c9-8e5b-792f310db524 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:44.897607 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c1408a9f-8983-43c9-8e5b-792f310db524 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:44.904059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1408a9f-8983-43c9-8e5b-792f310db524 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:44.904503 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1408a9f-8983-43c9-8e5b-792f310db524 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:44.905279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1408a9f-8983-43c9-8e5b-792f310db524 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:44.905710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1408a9f-8983-43c9-8e5b-792f310db524 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:45.028594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c1408a9f-8983-43c9-8e5b-792f310db524 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:45.036960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c1408a9f-8983-43c9-8e5b-792f310db524 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143869 Jun 10 13:28:45.037528 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1212/2126] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:44 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:46.054434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-22010230-ce19-4c66-84bd-b3fc8692b54a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:46.055087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-22010230-ce19-4c66-84bd-b3fc8692b54a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:46.061615 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-22010230-ce19-4c66-84bd-b3fc8692b54a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:46.062061 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-22010230-ce19-4c66-84bd-b3fc8692b54a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:46.062811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-22010230-ce19-4c66-84bd-b3fc8692b54a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:46.063863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-22010230-ce19-4c66-84bd-b3fc8692b54a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:46.185828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-22010230-ce19-4c66-84bd-b3fc8692b54a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:46.194224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-22010230-ce19-4c66-84bd-b3fc8692b54a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143734 Jun 10 13:28:46.194788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1213/2127] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:46 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:28:47.211113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f735a9a8-fe87-4459-b10b-7d0498dc169c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:47.211817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f735a9a8-fe87-4459-b10b-7d0498dc169c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:47.218455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f735a9a8-fe87-4459-b10b-7d0498dc169c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:47.219126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f735a9a8-fe87-4459-b10b-7d0498dc169c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:47.219738 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f735a9a8-fe87-4459-b10b-7d0498dc169c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:47.220165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f735a9a8-fe87-4459-b10b-7d0498dc169c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:47.349311 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f735a9a8-fe87-4459-b10b-7d0498dc169c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:47.357405 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f735a9a8-fe87-4459-b10b-7d0498dc169c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150168 Jun 10 13:28:47.357962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1214/2128] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:47 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:48.374908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-00c44232-82e5-47b7-a14d-06c4124d11b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:48.376170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-00c44232-82e5-47b7-a14d-06c4124d11b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:48.382720 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00c44232-82e5-47b7-a14d-06c4124d11b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:48.383170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00c44232-82e5-47b7-a14d-06c4124d11b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:48.383924 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00c44232-82e5-47b7-a14d-06c4124d11b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:48.384358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00c44232-82e5-47b7-a14d-06c4124d11b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:48.508433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-00c44232-82e5-47b7-a14d-06c4124d11b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:48.516376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-00c44232-82e5-47b7-a14d-06c4124d11b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145334 Jun 10 13:28:48.516963 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1215/2129] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:48 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:49.533314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c7aab272-0901-4384-b87f-eaf5f8ebd7d4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:49.533912 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c7aab272-0901-4384-b87f-eaf5f8ebd7d4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:49.540334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7aab272-0901-4384-b87f-eaf5f8ebd7d4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:49.540806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7aab272-0901-4384-b87f-eaf5f8ebd7d4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:49.541566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7aab272-0901-4384-b87f-eaf5f8ebd7d4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:49.542001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7aab272-0901-4384-b87f-eaf5f8ebd7d4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:49.671447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c7aab272-0901-4384-b87f-eaf5f8ebd7d4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:49.679411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c7aab272-0901-4384-b87f-eaf5f8ebd7d4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150006 Jun 10 13:28:49.679970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1216/2130] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:49 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:50.697485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1cf74aec-d255-4bf3-8dcb-7fbe32ff6087 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:50.698128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1cf74aec-d255-4bf3-8dcb-7fbe32ff6087 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:50.704539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1cf74aec-d255-4bf3-8dcb-7fbe32ff6087 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:50.705007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1cf74aec-d255-4bf3-8dcb-7fbe32ff6087 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:50.705750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1cf74aec-d255-4bf3-8dcb-7fbe32ff6087 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:50.706181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1cf74aec-d255-4bf3-8dcb-7fbe32ff6087 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:50.829361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1cf74aec-d255-4bf3-8dcb-7fbe32ff6087 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:50.837129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1cf74aec-d255-4bf3-8dcb-7fbe32ff6087 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143500 Jun 10 13:28:50.838137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1217/2131] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:50 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:51.855497 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-18651f8c-522a-4250-9ca0-ca6cf957b151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:51.856167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-18651f8c-522a-4250-9ca0-ca6cf957b151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:51.862711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18651f8c-522a-4250-9ca0-ca6cf957b151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:51.863203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18651f8c-522a-4250-9ca0-ca6cf957b151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:51.863953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18651f8c-522a-4250-9ca0-ca6cf957b151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:51.864512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18651f8c-522a-4250-9ca0-ca6cf957b151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:51.995121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-18651f8c-522a-4250-9ca0-ca6cf957b151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:52.003630 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-18651f8c-522a-4250-9ca0-ca6cf957b151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.152433 Jun 10 13:28:52.004192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1218/2132] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:51 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:53.021047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ad54b9da-9fba-4363-a7fc-5cf45f7aa3b0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:53.021684 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ad54b9da-9fba-4363-a7fc-5cf45f7aa3b0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:53.028023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad54b9da-9fba-4363-a7fc-5cf45f7aa3b0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:53.028467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad54b9da-9fba-4363-a7fc-5cf45f7aa3b0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:53.029243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad54b9da-9fba-4363-a7fc-5cf45f7aa3b0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:53.029672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad54b9da-9fba-4363-a7fc-5cf45f7aa3b0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:53.159887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ad54b9da-9fba-4363-a7fc-5cf45f7aa3b0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:53.167801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ad54b9da-9fba-4363-a7fc-5cf45f7aa3b0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.151207 Jun 10 13:28:53.168362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1219/2133] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:53 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:54.185243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-90c80009-e15d-482e-87a1-126d5b153e2c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:54.185923 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-90c80009-e15d-482e-87a1-126d5b153e2c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:54.192558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90c80009-e15d-482e-87a1-126d5b153e2c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:54.193035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90c80009-e15d-482e-87a1-126d5b153e2c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:54.193785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90c80009-e15d-482e-87a1-126d5b153e2c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:54.194214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90c80009-e15d-482e-87a1-126d5b153e2c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:54.323161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-90c80009-e15d-482e-87a1-126d5b153e2c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:54.331581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-90c80009-e15d-482e-87a1-126d5b153e2c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150156 Jun 10 13:28:54.332139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1220/2134] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:54 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:55.349322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-645698e5-5d23-41bc-bc2d-ccf393f2e35c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:55.349936 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-645698e5-5d23-41bc-bc2d-ccf393f2e35c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:55.356536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-645698e5-5d23-41bc-bc2d-ccf393f2e35c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:55.357102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-645698e5-5d23-41bc-bc2d-ccf393f2e35c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:55.357865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-645698e5-5d23-41bc-bc2d-ccf393f2e35c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:55.358303 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-645698e5-5d23-41bc-bc2d-ccf393f2e35c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:55.485945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-645698e5-5d23-41bc-bc2d-ccf393f2e35c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:55.493904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-645698e5-5d23-41bc-bc2d-ccf393f2e35c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149068 Jun 10 13:28:55.494458 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1221/2135] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:55 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:56.511371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-98db63a7-afba-4d9f-9a63-42e9f5ee94a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:56.511978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-98db63a7-afba-4d9f-9a63-42e9f5ee94a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:56.514414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=84 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:28:56.519168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98db63a7-afba-4d9f-9a63-42e9f5ee94a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:56.519621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98db63a7-afba-4d9f-9a63-42e9f5ee94a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:56.520385 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98db63a7-afba-4d9f-9a63-42e9f5ee94a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:56.520847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98db63a7-afba-4d9f-9a63-42e9f5ee94a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:56.524754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:28:56.645881 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-98db63a7-afba-4d9f-9a63-42e9f5ee94a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:56.653854 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-98db63a7-afba-4d9f-9a63-42e9f5ee94a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146398 Jun 10 13:28:56.654412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1222/2136] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:56 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:57.670392 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-41ac52a2-be8f-4f01-8f64-7b8f45f2d965 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:57.670991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-41ac52a2-be8f-4f01-8f64-7b8f45f2d965 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:57.678072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-41ac52a2-be8f-4f01-8f64-7b8f45f2d965 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:57.678518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-41ac52a2-be8f-4f01-8f64-7b8f45f2d965 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:57.679376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-41ac52a2-be8f-4f01-8f64-7b8f45f2d965 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:57.679817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-41ac52a2-be8f-4f01-8f64-7b8f45f2d965 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:57.806356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-41ac52a2-be8f-4f01-8f64-7b8f45f2d965 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:57.814294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-41ac52a2-be8f-4f01-8f64-7b8f45f2d965 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147853 Jun 10 13:28:57.814854 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1223/2137] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:57 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:58.831042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a0c9ffe9-3693-4b68-bb40-32a21ce945fc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:58.831703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a0c9ffe9-3693-4b68-bb40-32a21ce945fc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:28:58.838412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0c9ffe9-3693-4b68-bb40-32a21ce945fc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:58.838863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0c9ffe9-3693-4b68-bb40-32a21ce945fc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:58.839650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0c9ffe9-3693-4b68-bb40-32a21ce945fc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:28:58.840088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0c9ffe9-3693-4b68-bb40-32a21ce945fc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:28:58.971113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a0c9ffe9-3693-4b68-bb40-32a21ce945fc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:28:58.980835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a0c9ffe9-3693-4b68-bb40-32a21ce945fc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.151952 Jun 10 13:28:58.980835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1224/2138] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:58 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:28:59.996333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-583fe0e8-939c-44f8-b666-dd1a118fa7f8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:28:59.997040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-583fe0e8-939c-44f8-b666-dd1a118fa7f8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:00.003445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-583fe0e8-939c-44f8-b666-dd1a118fa7f8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:00.003897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-583fe0e8-939c-44f8-b666-dd1a118fa7f8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:00.004662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-583fe0e8-939c-44f8-b666-dd1a118fa7f8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:00.005122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-583fe0e8-939c-44f8-b666-dd1a118fa7f8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:00.129383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-583fe0e8-939c-44f8-b666-dd1a118fa7f8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:00.137304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-583fe0e8-939c-44f8-b666-dd1a118fa7f8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144804 Jun 10 13:29:00.137872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1225/2139] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:28:59 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:01.154342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a1bf1499-3a3c-4adc-a634-602e91479ed1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:01.155014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a1bf1499-3a3c-4adc-a634-602e91479ed1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:01.161736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a1bf1499-3a3c-4adc-a634-602e91479ed1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:01.162182 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a1bf1499-3a3c-4adc-a634-602e91479ed1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:01.162926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a1bf1499-3a3c-4adc-a634-602e91479ed1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:01.163355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a1bf1499-3a3c-4adc-a634-602e91479ed1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:01.293915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a1bf1499-3a3c-4adc-a634-602e91479ed1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:01.301736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a1bf1499-3a3c-4adc-a634-602e91479ed1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.151258 Jun 10 13:29:01.302298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1226/2140] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:01 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:02.319064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f5674d77-651d-4d30-92b5-c2b8c5da5ff9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:02.319712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f5674d77-651d-4d30-92b5-c2b8c5da5ff9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:02.326279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f5674d77-651d-4d30-92b5-c2b8c5da5ff9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:02.326721 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f5674d77-651d-4d30-92b5-c2b8c5da5ff9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:02.327473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f5674d77-651d-4d30-92b5-c2b8c5da5ff9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:02.327903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f5674d77-651d-4d30-92b5-c2b8c5da5ff9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:02.455493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f5674d77-651d-4d30-92b5-c2b8c5da5ff9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:02.463633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f5674d77-651d-4d30-92b5-c2b8c5da5ff9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148380 Jun 10 13:29:02.464195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1227/2141] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:02 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:03.481034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-649910f7-cf86-455d-aa8b-e33df4980432 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:03.481632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-649910f7-cf86-455d-aa8b-e33df4980432 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:03.488148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-649910f7-cf86-455d-aa8b-e33df4980432 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:03.488594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-649910f7-cf86-455d-aa8b-e33df4980432 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:03.489385 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-649910f7-cf86-455d-aa8b-e33df4980432 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:03.489820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-649910f7-cf86-455d-aa8b-e33df4980432 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:03.621856 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-649910f7-cf86-455d-aa8b-e33df4980432 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:03.630260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-649910f7-cf86-455d-aa8b-e33df4980432 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.153167 Jun 10 13:29:03.630827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1228/2142] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:03 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:04.647374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-36d62ef2-4acc-482e-9389-a13d7aefb9e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:04.647961 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-36d62ef2-4acc-482e-9389-a13d7aefb9e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:04.654349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36d62ef2-4acc-482e-9389-a13d7aefb9e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:04.654793 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36d62ef2-4acc-482e-9389-a13d7aefb9e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:04.655539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36d62ef2-4acc-482e-9389-a13d7aefb9e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:04.655967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-36d62ef2-4acc-482e-9389-a13d7aefb9e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:04.779421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-36d62ef2-4acc-482e-9389-a13d7aefb9e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:04.787879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-36d62ef2-4acc-482e-9389-a13d7aefb9e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144398 Jun 10 13:29:04.788457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1229/2143] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:04 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:05.804858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b78ac2d3-4d95-401b-a489-a3ab5ab205a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:05.805511 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b78ac2d3-4d95-401b-a489-a3ab5ab205a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:05.811897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b78ac2d3-4d95-401b-a489-a3ab5ab205a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:05.812343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b78ac2d3-4d95-401b-a489-a3ab5ab205a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:05.813727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b78ac2d3-4d95-401b-a489-a3ab5ab205a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:05.814166 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b78ac2d3-4d95-401b-a489-a3ab5ab205a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:05.939603 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b78ac2d3-4d95-401b-a489-a3ab5ab205a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:05.947472 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b78ac2d3-4d95-401b-a489-a3ab5ab205a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146489 Jun 10 13:29:05.948030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1230/2144] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:05 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:06.964013 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-683e7e6e-47f6-4cea-b469-df8becb5b8bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:06.964675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-683e7e6e-47f6-4cea-b469-df8becb5b8bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:06.971106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-683e7e6e-47f6-4cea-b469-df8becb5b8bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:06.971555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-683e7e6e-47f6-4cea-b469-df8becb5b8bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:06.972317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-683e7e6e-47f6-4cea-b469-df8becb5b8bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:06.972754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-683e7e6e-47f6-4cea-b469-df8becb5b8bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:07.101018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-683e7e6e-47f6-4cea-b469-df8becb5b8bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:07.108902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-683e7e6e-47f6-4cea-b469-df8becb5b8bc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149014 Jun 10 13:29:07.109966 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1231/2145] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:06 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:08.126237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bc1573e2-00ab-4e5f-a99e-e1fb749748b1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:08.126929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bc1573e2-00ab-4e5f-a99e-e1fb749748b1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:08.133318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bc1573e2-00ab-4e5f-a99e-e1fb749748b1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:08.133778 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bc1573e2-00ab-4e5f-a99e-e1fb749748b1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:08.134531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bc1573e2-00ab-4e5f-a99e-e1fb749748b1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:08.134964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bc1573e2-00ab-4e5f-a99e-e1fb749748b1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:08.268563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bc1573e2-00ab-4e5f-a99e-e1fb749748b1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:08.276539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bc1573e2-00ab-4e5f-a99e-e1fb749748b1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.154682 Jun 10 13:29:08.277146 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1232/2146] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:08 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:09.292956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9f9a980b-e8e0-4740-a663-567caeb1ba0d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:09.293636 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9f9a980b-e8e0-4740-a663-567caeb1ba0d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:09.300066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9f9a980b-e8e0-4740-a663-567caeb1ba0d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:09.300514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9f9a980b-e8e0-4740-a663-567caeb1ba0d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:09.301289 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9f9a980b-e8e0-4740-a663-567caeb1ba0d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:09.301723 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9f9a980b-e8e0-4740-a663-567caeb1ba0d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:09.428401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9f9a980b-e8e0-4740-a663-567caeb1ba0d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:09.436281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9f9a980b-e8e0-4740-a663-567caeb1ba0d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147244 Jun 10 13:29:09.436867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1233/2147] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:09 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:10.453497 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a68e01d0-0251-4dfd-80c1-03e38f5a7508 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:10.454134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a68e01d0-0251-4dfd-80c1-03e38f5a7508 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:10.460624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a68e01d0-0251-4dfd-80c1-03e38f5a7508 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:10.461108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a68e01d0-0251-4dfd-80c1-03e38f5a7508 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:10.462128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a68e01d0-0251-4dfd-80c1-03e38f5a7508 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:10.462570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a68e01d0-0251-4dfd-80c1-03e38f5a7508 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:10.584903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a68e01d0-0251-4dfd-80c1-03e38f5a7508 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:10.593100 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a68e01d0-0251-4dfd-80c1-03e38f5a7508 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143477 Jun 10 13:29:10.593605 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1234/2148] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:10 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:11.610719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-44d2c474-fe6a-49db-b332-932221d83b58 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:11.611357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-44d2c474-fe6a-49db-b332-932221d83b58 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:11.617934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-44d2c474-fe6a-49db-b332-932221d83b58 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:11.618373 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-44d2c474-fe6a-49db-b332-932221d83b58 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:11.619120 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-44d2c474-fe6a-49db-b332-932221d83b58 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:11.619548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-44d2c474-fe6a-49db-b332-932221d83b58 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:11.745819 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-44d2c474-fe6a-49db-b332-932221d83b58 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:11.753655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-44d2c474-fe6a-49db-b332-932221d83b58 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147015 Jun 10 13:29:11.754220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1235/2149] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:11 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:12.770946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8f276cfc-be96-4bae-b3c9-925d8328f246 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:12.771538 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8f276cfc-be96-4bae-b3c9-925d8328f246 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:12.778141 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f276cfc-be96-4bae-b3c9-925d8328f246 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:12.778582 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f276cfc-be96-4bae-b3c9-925d8328f246 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:12.779326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f276cfc-be96-4bae-b3c9-925d8328f246 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:12.779754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f276cfc-be96-4bae-b3c9-925d8328f246 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:12.913799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8f276cfc-be96-4bae-b3c9-925d8328f246 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:12.921656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8f276cfc-be96-4bae-b3c9-925d8328f246 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.154576 Jun 10 13:29:12.922215 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1236/2150] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:12 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:13.939727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c9af0366-8b30-49e2-964f-c7ad1a6ea1de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:13.940419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c9af0366-8b30-49e2-964f-c7ad1a6ea1de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:13.946830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9af0366-8b30-49e2-964f-c7ad1a6ea1de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:13.947502 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9af0366-8b30-49e2-964f-c7ad1a6ea1de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:13.948253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9af0366-8b30-49e2-964f-c7ad1a6ea1de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:13.948684 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9af0366-8b30-49e2-964f-c7ad1a6ea1de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:14.075928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c9af0366-8b30-49e2-964f-c7ad1a6ea1de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:14.083928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c9af0366-8b30-49e2-964f-c7ad1a6ea1de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148667 Jun 10 13:29:14.084494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1237/2151] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:13 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:15.101153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-efdc9169-b191-4e36-96a9-d206b5aaf3b9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:15.101818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-efdc9169-b191-4e36-96a9-d206b5aaf3b9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:15.108345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-efdc9169-b191-4e36-96a9-d206b5aaf3b9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:15.108815 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-efdc9169-b191-4e36-96a9-d206b5aaf3b9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:15.109625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-efdc9169-b191-4e36-96a9-d206b5aaf3b9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:15.110054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-efdc9169-b191-4e36-96a9-d206b5aaf3b9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:15.237453 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-efdc9169-b191-4e36-96a9-d206b5aaf3b9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:15.240813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:29:15.245499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-efdc9169-b191-4e36-96a9-d206b5aaf3b9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148337 Jun 10 13:29:15.246053 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1238/2152] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:15 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:15.273993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9c7affd1-9349-4b27-84fb-6b8ae99f1506 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:15.274473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9c7affd1-9349-4b27-84fb-6b8ae99f1506 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:15.280293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c7affd1-9349-4b27-84fb-6b8ae99f1506 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:15.280735 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c7affd1-9349-4b27-84fb-6b8ae99f1506 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:15.281514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c7affd1-9349-4b27-84fb-6b8ae99f1506 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:15.281941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9c7affd1-9349-4b27-84fb-6b8ae99f1506 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:15.305108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9c7affd1-9349-4b27-84fb-6b8ae99f1506 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464/ips" status: 200 len: 153 microversion: 2.20 time: 0.034644 Jun 10 13:29:15.305665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1239/2153] 10.222.0.25 () {60 vars in 1288 bytes} [Fri Jun 10 13:29:15 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464/ips => generated 153 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jun 10 13:29:16.262803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-29dc85bd-31fb-442f-a7cd-3ff5b4e0b9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:16.263496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-29dc85bd-31fb-442f-a7cd-3ff5b4e0b9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:16.270559 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-29dc85bd-31fb-442f-a7cd-3ff5b4e0b9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:16.271033 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-29dc85bd-31fb-442f-a7cd-3ff5b4e0b9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:16.271798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-29dc85bd-31fb-442f-a7cd-3ff5b4e0b9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:16.272233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-29dc85bd-31fb-442f-a7cd-3ff5b4e0b9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:16.401344 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-29dc85bd-31fb-442f-a7cd-3ff5b4e0b9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:16.409266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-29dc85bd-31fb-442f-a7cd-3ff5b4e0b9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150258 Jun 10 13:29:16.409833 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1240/2154] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:16 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:29:17.426209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-01ad8dee-40d4-4252-8632-370addc94489 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:17.426815 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-01ad8dee-40d4-4252-8632-370addc94489 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:17.433304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01ad8dee-40d4-4252-8632-370addc94489 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:17.433752 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01ad8dee-40d4-4252-8632-370addc94489 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:17.434510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01ad8dee-40d4-4252-8632-370addc94489 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:17.434945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01ad8dee-40d4-4252-8632-370addc94489 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:17.560718 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-01ad8dee-40d4-4252-8632-370addc94489 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:17.569252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-01ad8dee-40d4-4252-8632-370addc94489 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147467 Jun 10 13:29:17.569811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1241/2155] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:17 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:18.586305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-10bc837f-86cf-43df-aabb-875f075373a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:18.587097 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-10bc837f-86cf-43df-aabb-875f075373a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:18.593576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10bc837f-86cf-43df-aabb-875f075373a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:18.594027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10bc837f-86cf-43df-aabb-875f075373a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:18.594781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10bc837f-86cf-43df-aabb-875f075373a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:18.595216 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-10bc837f-86cf-43df-aabb-875f075373a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:18.717616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-10bc837f-86cf-43df-aabb-875f075373a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:18.725526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-10bc837f-86cf-43df-aabb-875f075373a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143076 Jun 10 13:29:18.726092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1242/2156] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:18 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:19.751969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-162af1d0-3cc7-4251-bfac-fc1348c0fb91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:19.752596 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-162af1d0-3cc7-4251-bfac-fc1348c0fb91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:19.759030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-162af1d0-3cc7-4251-bfac-fc1348c0fb91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:19.759478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-162af1d0-3cc7-4251-bfac-fc1348c0fb91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:19.760237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-162af1d0-3cc7-4251-bfac-fc1348c0fb91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:19.760668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-162af1d0-3cc7-4251-bfac-fc1348c0fb91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:19.885793 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-162af1d0-3cc7-4251-bfac-fc1348c0fb91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:19.894256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-162af1d0-3cc7-4251-bfac-fc1348c0fb91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146760 Jun 10 13:29:19.894826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1243/2157] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:19 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:20.911418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-55e60127-2258-49d1-9cd5-6fd44265c1d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:20.912001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-55e60127-2258-49d1-9cd5-6fd44265c1d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:20.918451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55e60127-2258-49d1-9cd5-6fd44265c1d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:20.918900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55e60127-2258-49d1-9cd5-6fd44265c1d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:20.919656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55e60127-2258-49d1-9cd5-6fd44265c1d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:20.920092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55e60127-2258-49d1-9cd5-6fd44265c1d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:21.046664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-55e60127-2258-49d1-9cd5-6fd44265c1d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:21.054526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-55e60127-2258-49d1-9cd5-6fd44265c1d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147084 Jun 10 13:29:21.055086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1244/2158] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:20 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:22.071414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-49817022-3d9c-4e41-95f4-1ca62f4d7773 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:22.072144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-49817022-3d9c-4e41-95f4-1ca62f4d7773 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:22.078295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-49817022-3d9c-4e41-95f4-1ca62f4d7773 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:22.078734 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-49817022-3d9c-4e41-95f4-1ca62f4d7773 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:22.079478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-49817022-3d9c-4e41-95f4-1ca62f4d7773 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:22.079943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-49817022-3d9c-4e41-95f4-1ca62f4d7773 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:22.207610 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-49817022-3d9c-4e41-95f4-1ca62f4d7773 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:22.215791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-49817022-3d9c-4e41-95f4-1ca62f4d7773 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148282 Jun 10 13:29:22.218322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1245/2159] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:22 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:23.232985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1294f790-8c56-4bc5-84f9-8c97cd976e95 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:23.233668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1294f790-8c56-4bc5-84f9-8c97cd976e95 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:23.240198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1294f790-8c56-4bc5-84f9-8c97cd976e95 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:23.240647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1294f790-8c56-4bc5-84f9-8c97cd976e95 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:23.241439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1294f790-8c56-4bc5-84f9-8c97cd976e95 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:23.241877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1294f790-8c56-4bc5-84f9-8c97cd976e95 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:23.563025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1294f790-8c56-4bc5-84f9-8c97cd976e95 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:23.570872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1294f790-8c56-4bc5-84f9-8c97cd976e95 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.341846 Jun 10 13:29:23.571425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1246/2160] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:23 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:24.587801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-be82cf95-2483-44f4-ab68-543f97c8cef7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:24.588440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-be82cf95-2483-44f4-ab68-543f97c8cef7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:24.594871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be82cf95-2483-44f4-ab68-543f97c8cef7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:24.595830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be82cf95-2483-44f4-ab68-543f97c8cef7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:24.596594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be82cf95-2483-44f4-ab68-543f97c8cef7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:24.597057 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be82cf95-2483-44f4-ab68-543f97c8cef7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:24.724631 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-be82cf95-2483-44f4-ab68-543f97c8cef7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:24.732525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-be82cf95-2483-44f4-ab68-543f97c8cef7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148985 Jun 10 13:29:24.733159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1247/2161] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:24 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:25.750435 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-18e913ed-a5d5-449c-8244-456388a7310b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:25.751071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-18e913ed-a5d5-449c-8244-456388a7310b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:25.757522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18e913ed-a5d5-449c-8244-456388a7310b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:25.757969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18e913ed-a5d5-449c-8244-456388a7310b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:25.758719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18e913ed-a5d5-449c-8244-456388a7310b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:25.759155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18e913ed-a5d5-449c-8244-456388a7310b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:25.883525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-18e913ed-a5d5-449c-8244-456388a7310b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:25.891347 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-18e913ed-a5d5-449c-8244-456388a7310b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144857 Jun 10 13:29:25.891906 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1248/2162] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:25 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:26.909121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-968904f8-2650-4b7b-b862-3d428716cd1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:26.909756 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-968904f8-2650-4b7b-b862-3d428716cd1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:26.912074 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=54 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:29:26.916779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-968904f8-2650-4b7b-b862-3d428716cd1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:26.918865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-968904f8-2650-4b7b-b862-3d428716cd1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:26.918865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-968904f8-2650-4b7b-b862-3d428716cd1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:26.918865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-968904f8-2650-4b7b-b862-3d428716cd1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:27.041686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-968904f8-2650-4b7b-b862-3d428716cd1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:27.049588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-968904f8-2650-4b7b-b862-3d428716cd1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144546 Jun 10 13:29:27.050153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1249/2163] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:26 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:28.066463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-448146f4-30da-400c-85d6-1bcabf39d828 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:28.067138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-448146f4-30da-400c-85d6-1bcabf39d828 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:28.073562 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-448146f4-30da-400c-85d6-1bcabf39d828 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:28.074012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-448146f4-30da-400c-85d6-1bcabf39d828 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:28.074768 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-448146f4-30da-400c-85d6-1bcabf39d828 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:28.075209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-448146f4-30da-400c-85d6-1bcabf39d828 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:28.198967 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-448146f4-30da-400c-85d6-1bcabf39d828 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:28.207425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-448146f4-30da-400c-85d6-1bcabf39d828 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144842 Jun 10 13:29:28.207985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1250/2164] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:28 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:29.225132 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-acd63904-6f0d-4f64-bc20-d907bd65c077 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:29.225779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-acd63904-6f0d-4f64-bc20-d907bd65c077 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:29.232184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acd63904-6f0d-4f64-bc20-d907bd65c077 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:29.232628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acd63904-6f0d-4f64-bc20-d907bd65c077 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:29.233417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acd63904-6f0d-4f64-bc20-d907bd65c077 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:29.233860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acd63904-6f0d-4f64-bc20-d907bd65c077 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:29.361757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-acd63904-6f0d-4f64-bc20-d907bd65c077 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:29.369587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-acd63904-6f0d-4f64-bc20-d907bd65c077 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148298 Jun 10 13:29:29.370147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1251/2165] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:29 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:30.387422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4b6ec4d8-4dbb-4eca-bb0f-56b0f5a949ca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:30.388053 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4b6ec4d8-4dbb-4eca-bb0f-56b0f5a949ca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:30.394772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b6ec4d8-4dbb-4eca-bb0f-56b0f5a949ca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:30.395220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b6ec4d8-4dbb-4eca-bb0f-56b0f5a949ca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:30.395981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b6ec4d8-4dbb-4eca-bb0f-56b0f5a949ca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:30.396413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4b6ec4d8-4dbb-4eca-bb0f-56b0f5a949ca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:30.517932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4b6ec4d8-4dbb-4eca-bb0f-56b0f5a949ca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:30.526022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4b6ec4d8-4dbb-4eca-bb0f-56b0f5a949ca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.142415 Jun 10 13:29:30.526611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1252/2166] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:30 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:31.543415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f34f84d2-8df8-4ee3-bce8-d4122d64dd2a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:31.544060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f34f84d2-8df8-4ee3-bce8-d4122d64dd2a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:31.550324 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f34f84d2-8df8-4ee3-bce8-d4122d64dd2a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:31.550766 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f34f84d2-8df8-4ee3-bce8-d4122d64dd2a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:31.551517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f34f84d2-8df8-4ee3-bce8-d4122d64dd2a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:31.551946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f34f84d2-8df8-4ee3-bce8-d4122d64dd2a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:31.681179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f34f84d2-8df8-4ee3-bce8-d4122d64dd2a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:31.689727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f34f84d2-8df8-4ee3-bce8-d4122d64dd2a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150165 Jun 10 13:29:31.690285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1253/2167] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:31 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:32.707112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3c4eb4e8-3fe5-4d52-b3cb-8158ccb17d9e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:32.707739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3c4eb4e8-3fe5-4d52-b3cb-8158ccb17d9e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:32.714433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c4eb4e8-3fe5-4d52-b3cb-8158ccb17d9e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:32.714884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c4eb4e8-3fe5-4d52-b3cb-8158ccb17d9e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:32.715633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c4eb4e8-3fe5-4d52-b3cb-8158ccb17d9e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:32.716063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c4eb4e8-3fe5-4d52-b3cb-8158ccb17d9e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:32.838248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3c4eb4e8-3fe5-4d52-b3cb-8158ccb17d9e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:32.846074 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3c4eb4e8-3fe5-4d52-b3cb-8158ccb17d9e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143339 Jun 10 13:29:32.846641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1254/2168] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:32 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:33.862913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0954d4f4-1363-4638-9577-890f41e393ec tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:33.863603 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0954d4f4-1363-4638-9577-890f41e393ec tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:33.870081 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0954d4f4-1363-4638-9577-890f41e393ec tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:33.870525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0954d4f4-1363-4638-9577-890f41e393ec tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:33.871282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0954d4f4-1363-4638-9577-890f41e393ec tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:33.871704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0954d4f4-1363-4638-9577-890f41e393ec tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:34.000642 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0954d4f4-1363-4638-9577-890f41e393ec tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:34.002234 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:29:34.007021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0954d4f4-1363-4638-9577-890f41e393ec tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147972 Jun 10 13:29:34.007584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1255/2169] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:33 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:35.023682 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6d1f3320-6b4b-4faf-a705-b978c182f54f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:35.024276 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6d1f3320-6b4b-4faf-a705-b978c182f54f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:35.030812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d1f3320-6b4b-4faf-a705-b978c182f54f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:35.031255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d1f3320-6b4b-4faf-a705-b978c182f54f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:35.032006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d1f3320-6b4b-4faf-a705-b978c182f54f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:35.033126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d1f3320-6b4b-4faf-a705-b978c182f54f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:35.154419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6d1f3320-6b4b-4faf-a705-b978c182f54f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:35.162901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6d1f3320-6b4b-4faf-a705-b978c182f54f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143388 Jun 10 13:29:35.163458 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1256/2170] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:35 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:36.179929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c708d0f8-e9ea-4e5d-b32d-31b8e67dfbd4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:36.180609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c708d0f8-e9ea-4e5d-b32d-31b8e67dfbd4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:36.187001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c708d0f8-e9ea-4e5d-b32d-31b8e67dfbd4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:36.187449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c708d0f8-e9ea-4e5d-b32d-31b8e67dfbd4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:36.188203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c708d0f8-e9ea-4e5d-b32d-31b8e67dfbd4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:36.189253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c708d0f8-e9ea-4e5d-b32d-31b8e67dfbd4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:36.322123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c708d0f8-e9ea-4e5d-b32d-31b8e67dfbd4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:36.330064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c708d0f8-e9ea-4e5d-b32d-31b8e67dfbd4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.154055 Jun 10 13:29:36.330635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1257/2171] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:36 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:37.347491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fad2a0e7-b668-4ac6-99c8-c42717504ffe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:37.354349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fad2a0e7-b668-4ac6-99c8-c42717504ffe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:37.354686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fad2a0e7-b668-4ac6-99c8-c42717504ffe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:37.355130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fad2a0e7-b668-4ac6-99c8-c42717504ffe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:37.355879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fad2a0e7-b668-4ac6-99c8-c42717504ffe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:37.356308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fad2a0e7-b668-4ac6-99c8-c42717504ffe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:37.484435 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fad2a0e7-b668-4ac6-99c8-c42717504ffe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:37.492584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fad2a0e7-b668-4ac6-99c8-c42717504ffe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149091 Jun 10 13:29:37.493201 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1258/2172] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:37 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:38.510758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-342a0030-2c72-4370-b27a-4edc0b19ebe9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:38.511550 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-342a0030-2c72-4370-b27a-4edc0b19ebe9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:38.518209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-342a0030-2c72-4370-b27a-4edc0b19ebe9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:38.518656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-342a0030-2c72-4370-b27a-4edc0b19ebe9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:38.519421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-342a0030-2c72-4370-b27a-4edc0b19ebe9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:38.519854 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-342a0030-2c72-4370-b27a-4edc0b19ebe9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:38.648027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-342a0030-2c72-4370-b27a-4edc0b19ebe9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:38.655890 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-342a0030-2c72-4370-b27a-4edc0b19ebe9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149500 Jun 10 13:29:38.656443 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1259/2173] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:38 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:39.674365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e8cb7500-5411-42e9-b217-119d3fdf7145 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:39.674996 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e8cb7500-5411-42e9-b217-119d3fdf7145 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:39.681371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8cb7500-5411-42e9-b217-119d3fdf7145 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:39.686390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8cb7500-5411-42e9-b217-119d3fdf7145 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:39.686390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8cb7500-5411-42e9-b217-119d3fdf7145 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:39.686390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8cb7500-5411-42e9-b217-119d3fdf7145 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:39.809986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e8cb7500-5411-42e9-b217-119d3fdf7145 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:39.817887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e8cb7500-5411-42e9-b217-119d3fdf7145 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148060 Jun 10 13:29:39.818451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1260/2174] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:39 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:40.834535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-87a5bcbd-9a6c-43aa-8fa5-ddc33abcc882 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:40.835168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-87a5bcbd-9a6c-43aa-8fa5-ddc33abcc882 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:40.841898 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87a5bcbd-9a6c-43aa-8fa5-ddc33abcc882 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:40.842346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87a5bcbd-9a6c-43aa-8fa5-ddc33abcc882 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:40.843112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87a5bcbd-9a6c-43aa-8fa5-ddc33abcc882 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:40.843552 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-87a5bcbd-9a6c-43aa-8fa5-ddc33abcc882 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:40.974947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-87a5bcbd-9a6c-43aa-8fa5-ddc33abcc882 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:40.976144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-87a5bcbd-9a6c-43aa-8fa5-ddc33abcc882 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145466 Jun 10 13:29:40.976715 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1261/2175] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:40 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:41.992930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-22183a12-a6c4-4348-8940-b9e899d5d5c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:41.993591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-22183a12-a6c4-4348-8940-b9e899d5d5c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:41.999977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-22183a12-a6c4-4348-8940-b9e899d5d5c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:42.000426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-22183a12-a6c4-4348-8940-b9e899d5d5c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:42.001710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-22183a12-a6c4-4348-8940-b9e899d5d5c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:42.002141 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-22183a12-a6c4-4348-8940-b9e899d5d5c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:42.130038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-22183a12-a6c4-4348-8940-b9e899d5d5c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:42.137976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-22183a12-a6c4-4348-8940-b9e899d5d5c4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149032 Jun 10 13:29:42.138536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1262/2176] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:41 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:43.154989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-33ee618f-1ecd-46e9-9d3e-5de9f8f9af31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:43.155653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-33ee618f-1ecd-46e9-9d3e-5de9f8f9af31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:43.162784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33ee618f-1ecd-46e9-9d3e-5de9f8f9af31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:43.167824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33ee618f-1ecd-46e9-9d3e-5de9f8f9af31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:43.167824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33ee618f-1ecd-46e9-9d3e-5de9f8f9af31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:43.167824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-33ee618f-1ecd-46e9-9d3e-5de9f8f9af31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:43.293261 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-33ee618f-1ecd-46e9-9d3e-5de9f8f9af31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:43.301083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-33ee618f-1ecd-46e9-9d3e-5de9f8f9af31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149980 Jun 10 13:29:43.301652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1263/2177] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:43 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:44.318551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-712a7077-4d35-4778-a2bd-43e70fe6645c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:44.319203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-712a7077-4d35-4778-a2bd-43e70fe6645c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:44.325938 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-712a7077-4d35-4778-a2bd-43e70fe6645c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:44.326379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-712a7077-4d35-4778-a2bd-43e70fe6645c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:44.327138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-712a7077-4d35-4778-a2bd-43e70fe6645c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:44.327565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-712a7077-4d35-4778-a2bd-43e70fe6645c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:44.451676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-712a7077-4d35-4778-a2bd-43e70fe6645c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:44.459599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-712a7077-4d35-4778-a2bd-43e70fe6645c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144963 Jun 10 13:29:44.460169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1264/2178] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:44 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:45.479301 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8551df06-e33b-4224-a3c4-2096c9245f31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:45.479887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8551df06-e33b-4224-a3c4-2096c9245f31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:45.486383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8551df06-e33b-4224-a3c4-2096c9245f31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:45.486834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8551df06-e33b-4224-a3c4-2096c9245f31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:45.487571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8551df06-e33b-4224-a3c4-2096c9245f31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:45.487998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8551df06-e33b-4224-a3c4-2096c9245f31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:45.620313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8551df06-e33b-4224-a3c4-2096c9245f31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:45.628715 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8551df06-e33b-4224-a3c4-2096c9245f31 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.153222 Jun 10 13:29:45.629309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1265/2179] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:45 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:46.645587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6fc77044-20f8-461a-8485-cb219538f20f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:46.646198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6fc77044-20f8-461a-8485-cb219538f20f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:46.652664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6fc77044-20f8-461a-8485-cb219538f20f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:46.653210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6fc77044-20f8-461a-8485-cb219538f20f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:46.653964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6fc77044-20f8-461a-8485-cb219538f20f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:46.654428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6fc77044-20f8-461a-8485-cb219538f20f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:46.780173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6fc77044-20f8-461a-8485-cb219538f20f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:46.790519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6fc77044-20f8-461a-8485-cb219538f20f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148697 Jun 10 13:29:46.791109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1266/2180] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:46 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:47.807860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bbafb0a7-7ade-4858-b09a-f7b9c3a15785 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:47.808525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bbafb0a7-7ade-4858-b09a-f7b9c3a15785 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:47.814995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bbafb0a7-7ade-4858-b09a-f7b9c3a15785 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:47.815437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bbafb0a7-7ade-4858-b09a-f7b9c3a15785 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:47.816181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bbafb0a7-7ade-4858-b09a-f7b9c3a15785 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:47.816609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bbafb0a7-7ade-4858-b09a-f7b9c3a15785 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:47.943175 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bbafb0a7-7ade-4858-b09a-f7b9c3a15785 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:47.951065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bbafb0a7-7ade-4858-b09a-f7b9c3a15785 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147126 Jun 10 13:29:47.951625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1267/2181] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:47 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:48.968318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-614d9636-98cf-4ecd-94c1-aaba06bef1d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:48.969050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-614d9636-98cf-4ecd-94c1-aaba06bef1d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:48.975529 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-614d9636-98cf-4ecd-94c1-aaba06bef1d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:48.975971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-614d9636-98cf-4ecd-94c1-aaba06bef1d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:48.976749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-614d9636-98cf-4ecd-94c1-aaba06bef1d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:48.977201 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-614d9636-98cf-4ecd-94c1-aaba06bef1d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:49.100210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-614d9636-98cf-4ecd-94c1-aaba06bef1d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:49.107996 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-614d9636-98cf-4ecd-94c1-aaba06bef1d8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143546 Jun 10 13:29:49.108548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1268/2182] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:48 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:50.126526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e24fcb43-f190-45b7-9881-9d32b0c3197e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:50.127123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e24fcb43-f190-45b7-9881-9d32b0c3197e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:50.133555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e24fcb43-f190-45b7-9881-9d32b0c3197e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:50.134030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e24fcb43-f190-45b7-9881-9d32b0c3197e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:50.134754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e24fcb43-f190-45b7-9881-9d32b0c3197e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:50.135194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e24fcb43-f190-45b7-9881-9d32b0c3197e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:50.258679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e24fcb43-f190-45b7-9881-9d32b0c3197e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:50.267181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e24fcb43-f190-45b7-9881-9d32b0c3197e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145024 Jun 10 13:29:50.267739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1269/2183] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:50 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:51.284020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-efc8a26d-82d8-43ab-92cd-a49a14f95b60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:51.284675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-efc8a26d-82d8-43ab-92cd-a49a14f95b60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:51.291155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-efc8a26d-82d8-43ab-92cd-a49a14f95b60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:51.291599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-efc8a26d-82d8-43ab-92cd-a49a14f95b60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:51.292353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-efc8a26d-82d8-43ab-92cd-a49a14f95b60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:51.292809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-efc8a26d-82d8-43ab-92cd-a49a14f95b60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:51.423349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-efc8a26d-82d8-43ab-92cd-a49a14f95b60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:51.431579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-efc8a26d-82d8-43ab-92cd-a49a14f95b60 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.151388 Jun 10 13:29:51.432168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1270/2184] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:51 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:52.448574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9a9e9e00-2e1b-47b8-a3e5-67b26e0c180f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:52.449246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9a9e9e00-2e1b-47b8-a3e5-67b26e0c180f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:52.456649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a9e9e00-2e1b-47b8-a3e5-67b26e0c180f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:52.457119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a9e9e00-2e1b-47b8-a3e5-67b26e0c180f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:52.457880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a9e9e00-2e1b-47b8-a3e5-67b26e0c180f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:52.458307 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9a9e9e00-2e1b-47b8-a3e5-67b26e0c180f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:52.587286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9a9e9e00-2e1b-47b8-a3e5-67b26e0c180f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:52.595118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9a9e9e00-2e1b-47b8-a3e5-67b26e0c180f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150443 Jun 10 13:29:52.595666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1271/2185] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:52 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:53.612110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cc96cce8-33f0-418a-99d7-43aa34cb9c35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:53.612744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cc96cce8-33f0-418a-99d7-43aa34cb9c35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:53.619054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cc96cce8-33f0-418a-99d7-43aa34cb9c35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:53.619494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cc96cce8-33f0-418a-99d7-43aa34cb9c35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:53.620236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cc96cce8-33f0-418a-99d7-43aa34cb9c35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:53.620661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cc96cce8-33f0-418a-99d7-43aa34cb9c35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:53.637915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:29:53.748116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cc96cce8-33f0-418a-99d7-43aa34cb9c35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:53.755926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cc96cce8-33f0-418a-99d7-43aa34cb9c35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147800 Jun 10 13:29:53.756488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1272/2186] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:53 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:54.772777 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e587ebcf-aab2-4914-a2f6-2990a3b2e644 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:54.773414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e587ebcf-aab2-4914-a2f6-2990a3b2e644 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:54.779866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e587ebcf-aab2-4914-a2f6-2990a3b2e644 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:54.780309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e587ebcf-aab2-4914-a2f6-2990a3b2e644 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:54.781642 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e587ebcf-aab2-4914-a2f6-2990a3b2e644 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:54.782073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e587ebcf-aab2-4914-a2f6-2990a3b2e644 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:54.906190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e587ebcf-aab2-4914-a2f6-2990a3b2e644 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:54.914348 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e587ebcf-aab2-4914-a2f6-2990a3b2e644 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145391 Jun 10 13:29:54.914940 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1273/2187] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:54 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:55.931861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6e479d4b-8163-4617-b016-4356bfbda10d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:55.932531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6e479d4b-8163-4617-b016-4356bfbda10d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:55.938947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e479d4b-8163-4617-b016-4356bfbda10d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:55.939390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e479d4b-8163-4617-b016-4356bfbda10d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:55.940139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e479d4b-8163-4617-b016-4356bfbda10d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:55.940564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6e479d4b-8163-4617-b016-4356bfbda10d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:56.067699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6e479d4b-8163-4617-b016-4356bfbda10d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:56.075518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6e479d4b-8163-4617-b016-4356bfbda10d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147582 Jun 10 13:29:56.076077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1274/2188] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:55 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:57.092444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-665f5457-ebd9-4673-8eb7-da33ba3e8f4b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:57.093136 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-665f5457-ebd9-4673-8eb7-da33ba3e8f4b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:57.095368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=52 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:29:57.100054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-665f5457-ebd9-4673-8eb7-da33ba3e8f4b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:57.100505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-665f5457-ebd9-4673-8eb7-da33ba3e8f4b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:57.101300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-665f5457-ebd9-4673-8eb7-da33ba3e8f4b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:57.101734 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-665f5457-ebd9-4673-8eb7-da33ba3e8f4b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:57.229482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-665f5457-ebd9-4673-8eb7-da33ba3e8f4b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:57.237926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-665f5457-ebd9-4673-8eb7-da33ba3e8f4b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149287 Jun 10 13:29:57.243950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1275/2189] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:57 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:58.255478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c866a3c4-a1c8-4902-aee6-995dd151f921 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:58.256152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c866a3c4-a1c8-4902-aee6-995dd151f921 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:58.262645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c866a3c4-a1c8-4902-aee6-995dd151f921 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:58.263090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c866a3c4-a1c8-4902-aee6-995dd151f921 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:58.263847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c866a3c4-a1c8-4902-aee6-995dd151f921 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:58.264280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c866a3c4-a1c8-4902-aee6-995dd151f921 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:58.390754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c866a3c4-a1c8-4902-aee6-995dd151f921 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:58.398586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c866a3c4-a1c8-4902-aee6-995dd151f921 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147030 Jun 10 13:29:58.399150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1276/2190] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:58 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:29:59.415959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fedef8e2-edd5-43e3-b194-40bcaef17199 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:29:59.416586 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fedef8e2-edd5-43e3-b194-40bcaef17199 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:29:59.424332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fedef8e2-edd5-43e3-b194-40bcaef17199 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:59.424814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fedef8e2-edd5-43e3-b194-40bcaef17199 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:59.425571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fedef8e2-edd5-43e3-b194-40bcaef17199 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:29:59.426002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fedef8e2-edd5-43e3-b194-40bcaef17199 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:29:59.556053 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fedef8e2-edd5-43e3-b194-40bcaef17199 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:29:59.564988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fedef8e2-edd5-43e3-b194-40bcaef17199 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.152798 Jun 10 13:29:59.565550 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1277/2191] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:29:59 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:30:00.582191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-45d23c6e-b090-4e45-b9bb-178cb1898520 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:00.582818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-45d23c6e-b090-4e45-b9bb-178cb1898520 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:00.589613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45d23c6e-b090-4e45-b9bb-178cb1898520 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:00.589998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45d23c6e-b090-4e45-b9bb-178cb1898520 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:00.590756 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45d23c6e-b090-4e45-b9bb-178cb1898520 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:00.591182 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45d23c6e-b090-4e45-b9bb-178cb1898520 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:00.737968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-45d23c6e-b090-4e45-b9bb-178cb1898520 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:00.746025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-45d23c6e-b090-4e45-b9bb-178cb1898520 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.167703 Jun 10 13:30:00.746593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1278/2192] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:00 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:01.762913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0fa740f5-65b9-4bf0-a099-1e6806a0619e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:01.763576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0fa740f5-65b9-4bf0-a099-1e6806a0619e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:01.769981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fa740f5-65b9-4bf0-a099-1e6806a0619e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:01.770421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fa740f5-65b9-4bf0-a099-1e6806a0619e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:01.771167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fa740f5-65b9-4bf0-a099-1e6806a0619e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:01.771591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0fa740f5-65b9-4bf0-a099-1e6806a0619e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:01.903946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0fa740f5-65b9-4bf0-a099-1e6806a0619e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:01.912043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0fa740f5-65b9-4bf0-a099-1e6806a0619e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.153027 Jun 10 13:30:01.912610 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1279/2193] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:01 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:02.929427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-61ef029b-d54c-4c93-bcfd-7f85ad6b5442 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:02.930093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-61ef029b-d54c-4c93-bcfd-7f85ad6b5442 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:02.936477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61ef029b-d54c-4c93-bcfd-7f85ad6b5442 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:02.937466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61ef029b-d54c-4c93-bcfd-7f85ad6b5442 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:02.938223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61ef029b-d54c-4c93-bcfd-7f85ad6b5442 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:02.938656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61ef029b-d54c-4c93-bcfd-7f85ad6b5442 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:03.062939 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-61ef029b-d54c-4c93-bcfd-7f85ad6b5442 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:03.070853 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-61ef029b-d54c-4c93-bcfd-7f85ad6b5442 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145267 Jun 10 13:30:03.071424 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1280/2194] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:02 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:04.087807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3ae976ab-81df-4ee5-97cb-9a55bb90f187 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:04.088509 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3ae976ab-81df-4ee5-97cb-9a55bb90f187 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:04.095129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3ae976ab-81df-4ee5-97cb-9a55bb90f187 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:04.095574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3ae976ab-81df-4ee5-97cb-9a55bb90f187 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:04.096329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3ae976ab-81df-4ee5-97cb-9a55bb90f187 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:04.096759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3ae976ab-81df-4ee5-97cb-9a55bb90f187 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:04.218628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3ae976ab-81df-4ee5-97cb-9a55bb90f187 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:04.226509 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3ae976ab-81df-4ee5-97cb-9a55bb90f187 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.142848 Jun 10 13:30:04.227077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1281/2195] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:04 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:05.243958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-01e02164-3964-4851-b7b3-61d47f38ae46 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:05.244668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-01e02164-3964-4851-b7b3-61d47f38ae46 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:05.251477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01e02164-3964-4851-b7b3-61d47f38ae46 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:05.251930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01e02164-3964-4851-b7b3-61d47f38ae46 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:05.252690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01e02164-3964-4851-b7b3-61d47f38ae46 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:05.253151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01e02164-3964-4851-b7b3-61d47f38ae46 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:05.382766 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-01e02164-3964-4851-b7b3-61d47f38ae46 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:05.390631 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-01e02164-3964-4851-b7b3-61d47f38ae46 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150549 Jun 10 13:30:05.391187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1282/2196] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:05 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:06.408128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-18a352fb-1219-4fa5-b5bb-44a30b3ba8dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:06.408798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-18a352fb-1219-4fa5-b5bb-44a30b3ba8dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:06.415238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18a352fb-1219-4fa5-b5bb-44a30b3ba8dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:06.415690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18a352fb-1219-4fa5-b5bb-44a30b3ba8dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:06.416441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18a352fb-1219-4fa5-b5bb-44a30b3ba8dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:06.416906 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-18a352fb-1219-4fa5-b5bb-44a30b3ba8dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:06.541377 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-18a352fb-1219-4fa5-b5bb-44a30b3ba8dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:06.549324 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-18a352fb-1219-4fa5-b5bb-44a30b3ba8dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145596 Jun 10 13:30:06.549898 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1283/2197] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:06 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:07.566802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c6f65402-8438-465f-8459-8ffeec575be8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:07.567379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c6f65402-8438-465f-8459-8ffeec575be8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:07.573758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6f65402-8438-465f-8459-8ffeec575be8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:07.574199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6f65402-8438-465f-8459-8ffeec575be8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:07.574946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6f65402-8438-465f-8459-8ffeec575be8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:07.575372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6f65402-8438-465f-8459-8ffeec575be8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:07.704958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c6f65402-8438-465f-8459-8ffeec575be8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:07.713078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c6f65402-8438-465f-8459-8ffeec575be8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150153 Jun 10 13:30:07.713634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1284/2198] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:07 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:08.730822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7c13c36e-b17b-45c8-a9df-ce16bf35b8a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:08.731431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7c13c36e-b17b-45c8-a9df-ce16bf35b8a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:08.738438 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7c13c36e-b17b-45c8-a9df-ce16bf35b8a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:08.746077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7c13c36e-b17b-45c8-a9df-ce16bf35b8a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:08.746077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7c13c36e-b17b-45c8-a9df-ce16bf35b8a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:08.746077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7c13c36e-b17b-45c8-a9df-ce16bf35b8a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:08.866909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7c13c36e-b17b-45c8-a9df-ce16bf35b8a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:08.874905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7c13c36e-b17b-45c8-a9df-ce16bf35b8a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147985 Jun 10 13:30:08.875466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1285/2199] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:08 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:09.892564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-96cb8fdf-bff7-41dc-8277-af06637f5fcc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:09.893255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-96cb8fdf-bff7-41dc-8277-af06637f5fcc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:09.899731 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96cb8fdf-bff7-41dc-8277-af06637f5fcc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:09.900172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96cb8fdf-bff7-41dc-8277-af06637f5fcc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:09.900955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96cb8fdf-bff7-41dc-8277-af06637f5fcc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:09.901387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96cb8fdf-bff7-41dc-8277-af06637f5fcc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:10.024422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-96cb8fdf-bff7-41dc-8277-af06637f5fcc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:10.032268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-96cb8fdf-bff7-41dc-8277-af06637f5fcc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144295 Jun 10 13:30:10.032842 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1286/2200] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:09 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:11.049191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ad4b360b-a925-403a-a689-930ab80792e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:11.049845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ad4b360b-a925-403a-a689-930ab80792e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:11.056200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad4b360b-a925-403a-a689-930ab80792e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:11.056651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad4b360b-a925-403a-a689-930ab80792e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:11.057444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad4b360b-a925-403a-a689-930ab80792e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:11.057884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ad4b360b-a925-403a-a689-930ab80792e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:11.183684 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ad4b360b-a925-403a-a689-930ab80792e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:11.192079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ad4b360b-a925-403a-a689-930ab80792e1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146817 Jun 10 13:30:11.192643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1287/2201] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:11 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:12.209091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-79e6d953-ca6b-496b-b231-b681766e9722 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:12.210022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-79e6d953-ca6b-496b-b231-b681766e9722 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:12.216344 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79e6d953-ca6b-496b-b231-b681766e9722 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:12.216812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79e6d953-ca6b-496b-b231-b681766e9722 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:12.217572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79e6d953-ca6b-496b-b231-b681766e9722 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:12.218649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79e6d953-ca6b-496b-b231-b681766e9722 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:12.341376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-79e6d953-ca6b-496b-b231-b681766e9722 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:12.349878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-79e6d953-ca6b-496b-b231-b681766e9722 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144598 Jun 10 13:30:12.350439 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1288/2202] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:12 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:13.366985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-185b3320-0163-43ff-af32-322242db1b13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:13.367598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-185b3320-0163-43ff-af32-322242db1b13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:13.374774 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-185b3320-0163-43ff-af32-322242db1b13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:13.375224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-185b3320-0163-43ff-af32-322242db1b13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:13.375982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-185b3320-0163-43ff-af32-322242db1b13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:13.376413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-185b3320-0163-43ff-af32-322242db1b13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:13.380422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:30:13.503620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-185b3320-0163-43ff-af32-322242db1b13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:13.511401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-185b3320-0163-43ff-af32-322242db1b13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148342 Jun 10 13:30:13.512023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1289/2203] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:13 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:14.529110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d5e751e3-7bf1-4ab4-a7e1-97f602d49723 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:14.529760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d5e751e3-7bf1-4ab4-a7e1-97f602d49723 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:14.536285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5e751e3-7bf1-4ab4-a7e1-97f602d49723 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:14.536733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5e751e3-7bf1-4ab4-a7e1-97f602d49723 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:14.537516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5e751e3-7bf1-4ab4-a7e1-97f602d49723 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:14.537949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5e751e3-7bf1-4ab4-a7e1-97f602d49723 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:14.663832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d5e751e3-7bf1-4ab4-a7e1-97f602d49723 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:14.672348 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d5e751e3-7bf1-4ab4-a7e1-97f602d49723 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147143 Jun 10 13:30:14.672964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1290/2204] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:14 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:15.689569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d24e0bd8-d8f9-4bb8-851a-b2d55a0b6437 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:15.690189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d24e0bd8-d8f9-4bb8-851a-b2d55a0b6437 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:15.697210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d24e0bd8-d8f9-4bb8-851a-b2d55a0b6437 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:15.697655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d24e0bd8-d8f9-4bb8-851a-b2d55a0b6437 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:15.698412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d24e0bd8-d8f9-4bb8-851a-b2d55a0b6437 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:15.698838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d24e0bd8-d8f9-4bb8-851a-b2d55a0b6437 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:15.824609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d24e0bd8-d8f9-4bb8-851a-b2d55a0b6437 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:15.832407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d24e0bd8-d8f9-4bb8-851a-b2d55a0b6437 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146791 Jun 10 13:30:15.832995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1291/2205] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:15 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:16.849578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-81a1139f-d093-461f-a5c9-6a69a569fbc7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:16.850174 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-81a1139f-d093-461f-a5c9-6a69a569fbc7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:16.856864 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-81a1139f-d093-461f-a5c9-6a69a569fbc7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:16.857320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-81a1139f-d093-461f-a5c9-6a69a569fbc7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:16.858140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-81a1139f-d093-461f-a5c9-6a69a569fbc7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:16.858578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-81a1139f-d093-461f-a5c9-6a69a569fbc7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:16.985585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-81a1139f-d093-461f-a5c9-6a69a569fbc7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:16.994324 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-81a1139f-d093-461f-a5c9-6a69a569fbc7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148569 Jun 10 13:30:16.994897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1292/2206] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:16 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:18.011268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cfa64267-6d01-431c-bf41-1211d47730b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:18.011904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cfa64267-6d01-431c-bf41-1211d47730b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:18.018969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfa64267-6d01-431c-bf41-1211d47730b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:18.019413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfa64267-6d01-431c-bf41-1211d47730b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:18.020162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfa64267-6d01-431c-bf41-1211d47730b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:18.020633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cfa64267-6d01-431c-bf41-1211d47730b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:18.144931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cfa64267-6d01-431c-bf41-1211d47730b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:18.152798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cfa64267-6d01-431c-bf41-1211d47730b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145423 Jun 10 13:30:18.153364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1293/2207] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:18 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:19.169425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5e22ad2d-dd99-4cff-868f-1a61cb6ab603 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:19.170059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5e22ad2d-dd99-4cff-868f-1a61cb6ab603 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:19.176406 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e22ad2d-dd99-4cff-868f-1a61cb6ab603 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:19.176905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e22ad2d-dd99-4cff-868f-1a61cb6ab603 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:19.177665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e22ad2d-dd99-4cff-868f-1a61cb6ab603 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:19.178101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e22ad2d-dd99-4cff-868f-1a61cb6ab603 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:19.309596 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5e22ad2d-dd99-4cff-868f-1a61cb6ab603 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:19.317992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5e22ad2d-dd99-4cff-868f-1a61cb6ab603 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.152547 Jun 10 13:30:19.318639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1294/2208] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:19 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:20.335775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e1a5d6a8-0ba1-4904-803f-e3885720e37f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:20.336448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e1a5d6a8-0ba1-4904-803f-e3885720e37f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:20.342950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e1a5d6a8-0ba1-4904-803f-e3885720e37f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:20.343400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e1a5d6a8-0ba1-4904-803f-e3885720e37f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:20.344156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e1a5d6a8-0ba1-4904-803f-e3885720e37f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:20.344589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e1a5d6a8-0ba1-4904-803f-e3885720e37f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:20.473157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e1a5d6a8-0ba1-4904-803f-e3885720e37f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:20.477189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e1a5d6a8-0ba1-4904-803f-e3885720e37f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145496 Jun 10 13:30:20.477756 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1295/2209] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:20 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:21.494879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0d0baa12-0b4e-46c9-8595-1130ce42ba91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:21.495499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0d0baa12-0b4e-46c9-8595-1130ce42ba91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:21.502538 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0d0baa12-0b4e-46c9-8595-1130ce42ba91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:21.502982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0d0baa12-0b4e-46c9-8595-1130ce42ba91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:21.503738 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0d0baa12-0b4e-46c9-8595-1130ce42ba91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:21.504171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0d0baa12-0b4e-46c9-8595-1130ce42ba91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:21.635866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0d0baa12-0b4e-46c9-8595-1130ce42ba91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:21.644023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0d0baa12-0b4e-46c9-8595-1130ce42ba91 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.153042 Jun 10 13:30:21.646089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1296/2210] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:21 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:22.661580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-aafb2b90-ead6-4558-a624-e6625a09b6a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:22.662196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-aafb2b90-ead6-4558-a624-e6625a09b6a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:22.668505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aafb2b90-ead6-4558-a624-e6625a09b6a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:22.669001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aafb2b90-ead6-4558-a624-e6625a09b6a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:22.669754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aafb2b90-ead6-4558-a624-e6625a09b6a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:22.670789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-aafb2b90-ead6-4558-a624-e6625a09b6a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:22.801995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-aafb2b90-ead6-4558-a624-e6625a09b6a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:22.810001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-aafb2b90-ead6-4558-a624-e6625a09b6a3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.152390 Jun 10 13:30:22.810569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1297/2211] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:22 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:30:23.827078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-485e905e-2f03-43e4-9135-23241b46ab74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:23.827760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-485e905e-2f03-43e4-9135-23241b46ab74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:23.834442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-485e905e-2f03-43e4-9135-23241b46ab74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:23.834892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-485e905e-2f03-43e4-9135-23241b46ab74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:23.835645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-485e905e-2f03-43e4-9135-23241b46ab74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:23.836081 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-485e905e-2f03-43e4-9135-23241b46ab74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:23.967106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-485e905e-2f03-43e4-9135-23241b46ab74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:23.975039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-485e905e-2f03-43e4-9135-23241b46ab74 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.151859 Jun 10 13:30:23.975601 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1298/2212] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:23 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:24.992029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5c3bdb73-2dbf-47ac-a23f-929d119ec12e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:24.992691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5c3bdb73-2dbf-47ac-a23f-929d119ec12e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:24.999730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c3bdb73-2dbf-47ac-a23f-929d119ec12e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:25.000178 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c3bdb73-2dbf-47ac-a23f-929d119ec12e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:25.000964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c3bdb73-2dbf-47ac-a23f-929d119ec12e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:25.001429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c3bdb73-2dbf-47ac-a23f-929d119ec12e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:25.131931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5c3bdb73-2dbf-47ac-a23f-929d119ec12e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:25.140358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5c3bdb73-2dbf-47ac-a23f-929d119ec12e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.152189 Jun 10 13:30:25.141043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1299/2213] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:24 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:26.157440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-99dcb993-a7e6-48a7-b3cd-92f1509ffff4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:26.158111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-99dcb993-a7e6-48a7-b3cd-92f1509ffff4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:26.164572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99dcb993-a7e6-48a7-b3cd-92f1509ffff4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:26.165056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99dcb993-a7e6-48a7-b3cd-92f1509ffff4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:26.165814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99dcb993-a7e6-48a7-b3cd-92f1509ffff4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:26.166249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99dcb993-a7e6-48a7-b3cd-92f1509ffff4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:26.289121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-99dcb993-a7e6-48a7-b3cd-92f1509ffff4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:26.297304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-99dcb993-a7e6-48a7-b3cd-92f1509ffff4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143698 Jun 10 13:30:26.297873 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1300/2214] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:26 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:27.314811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5e8f04a3-ea3d-469a-9b32-d8bc6aa4c575 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:27.315471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5e8f04a3-ea3d-469a-9b32-d8bc6aa4c575 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:27.317704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=52 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:30:27.323699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e8f04a3-ea3d-469a-9b32-d8bc6aa4c575 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:27.323699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e8f04a3-ea3d-469a-9b32-d8bc6aa4c575 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:27.323699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e8f04a3-ea3d-469a-9b32-d8bc6aa4c575 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:27.324095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5e8f04a3-ea3d-469a-9b32-d8bc6aa4c575 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:27.452645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5e8f04a3-ea3d-469a-9b32-d8bc6aa4c575 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:27.460874 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5e8f04a3-ea3d-469a-9b32-d8bc6aa4c575 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149909 Jun 10 13:30:27.461450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1301/2215] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:27 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:28.478416 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1c48b9e0-ef5f-42d0-adfc-8f776d7b5429 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:28.479065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1c48b9e0-ef5f-42d0-adfc-8f776d7b5429 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:28.486119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c48b9e0-ef5f-42d0-adfc-8f776d7b5429 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:28.486566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c48b9e0-ef5f-42d0-adfc-8f776d7b5429 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:28.487319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c48b9e0-ef5f-42d0-adfc-8f776d7b5429 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:28.487755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c48b9e0-ef5f-42d0-adfc-8f776d7b5429 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:28.611350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1c48b9e0-ef5f-42d0-adfc-8f776d7b5429 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:28.619259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1c48b9e0-ef5f-42d0-adfc-8f776d7b5429 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144684 Jun 10 13:30:28.619822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1302/2216] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:28 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:30:29.636155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-96b53173-4b59-4588-900d-2acb011ebec9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:29.636825 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-96b53173-4b59-4588-900d-2acb011ebec9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:29.643286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96b53173-4b59-4588-900d-2acb011ebec9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:29.643736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96b53173-4b59-4588-900d-2acb011ebec9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:29.644490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96b53173-4b59-4588-900d-2acb011ebec9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:29.644952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96b53173-4b59-4588-900d-2acb011ebec9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:29.773829 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-96b53173-4b59-4588-900d-2acb011ebec9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:29.782251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-96b53173-4b59-4588-900d-2acb011ebec9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149916 Jun 10 13:30:29.782820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1303/2217] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:29 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:30.799787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b315eae7-f34d-4aad-a7b1-cd972b0fee47 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:30.800426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b315eae7-f34d-4aad-a7b1-cd972b0fee47 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:30.806903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b315eae7-f34d-4aad-a7b1-cd972b0fee47 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:30.807355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b315eae7-f34d-4aad-a7b1-cd972b0fee47 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:30.808114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b315eae7-f34d-4aad-a7b1-cd972b0fee47 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:30.808549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b315eae7-f34d-4aad-a7b1-cd972b0fee47 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:30.931658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b315eae7-f34d-4aad-a7b1-cd972b0fee47 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:30.939552 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b315eae7-f34d-4aad-a7b1-cd972b0fee47 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143637 Jun 10 13:30:30.940117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1304/2218] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:30 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:31.955977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c75052cb-d249-4d53-a669-58d338b1433d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:31.956626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c75052cb-d249-4d53-a669-58d338b1433d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:31.963062 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c75052cb-d249-4d53-a669-58d338b1433d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:31.963510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c75052cb-d249-4d53-a669-58d338b1433d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:31.964267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c75052cb-d249-4d53-a669-58d338b1433d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:31.964700 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c75052cb-d249-4d53-a669-58d338b1433d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:32.088396 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c75052cb-d249-4d53-a669-58d338b1433d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:32.091785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:30:32.096546 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c75052cb-d249-4d53-a669-58d338b1433d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144404 Jun 10 13:30:32.097137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1305/2219] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:31 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:30:33.113809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9cb5a693-915f-4dc0-a0bc-0eb581828e73 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:33.114463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9cb5a693-915f-4dc0-a0bc-0eb581828e73 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:33.121059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cb5a693-915f-4dc0-a0bc-0eb581828e73 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:33.121505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cb5a693-915f-4dc0-a0bc-0eb581828e73 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:33.122350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cb5a693-915f-4dc0-a0bc-0eb581828e73 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:33.123335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cb5a693-915f-4dc0-a0bc-0eb581828e73 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:33.259295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9cb5a693-915f-4dc0-a0bc-0eb581828e73 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:33.267186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9cb5a693-915f-4dc0-a0bc-0eb581828e73 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.157720 Jun 10 13:30:33.267754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1306/2220] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:33 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:34.284002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9189e78e-41f9-4df4-a723-ead0dc5086e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:34.284633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9189e78e-41f9-4df4-a723-ead0dc5086e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:34.291025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9189e78e-41f9-4df4-a723-ead0dc5086e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:34.291466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9189e78e-41f9-4df4-a723-ead0dc5086e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:34.292207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9189e78e-41f9-4df4-a723-ead0dc5086e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:34.292635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9189e78e-41f9-4df4-a723-ead0dc5086e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:34.419421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9189e78e-41f9-4df4-a723-ead0dc5086e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:34.428049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9189e78e-41f9-4df4-a723-ead0dc5086e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148445 Jun 10 13:30:34.428610 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1307/2221] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:34 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:35.446074 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-236e607d-ed97-4f9d-91da-006ad00633be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:35.446687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-236e607d-ed97-4f9d-91da-006ad00633be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:35.453214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-236e607d-ed97-4f9d-91da-006ad00633be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:35.453658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-236e607d-ed97-4f9d-91da-006ad00633be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:35.454403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-236e607d-ed97-4f9d-91da-006ad00633be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:35.454832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-236e607d-ed97-4f9d-91da-006ad00633be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:35.589211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-236e607d-ed97-4f9d-91da-006ad00633be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:35.597153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-236e607d-ed97-4f9d-91da-006ad00633be tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.154950 Jun 10 13:30:35.597722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1308/2222] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:35 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:36.614183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a2065de1-134a-4a55-9420-d6e80342276f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:36.614781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a2065de1-134a-4a55-9420-d6e80342276f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:36.621263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2065de1-134a-4a55-9420-d6e80342276f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:36.621750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2065de1-134a-4a55-9420-d6e80342276f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:36.623003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2065de1-134a-4a55-9420-d6e80342276f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:36.623437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a2065de1-134a-4a55-9420-d6e80342276f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:36.751609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a2065de1-134a-4a55-9420-d6e80342276f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:36.759467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a2065de1-134a-4a55-9420-d6e80342276f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149264 Jun 10 13:30:36.760031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1309/2223] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:36 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:37.777305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a107411b-2bf2-45e3-b6ec-49a1b7cb963b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:37.778018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a107411b-2bf2-45e3-b6ec-49a1b7cb963b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:37.784349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a107411b-2bf2-45e3-b6ec-49a1b7cb963b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:37.784825 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a107411b-2bf2-45e3-b6ec-49a1b7cb963b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:37.785602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a107411b-2bf2-45e3-b6ec-49a1b7cb963b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:37.786031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a107411b-2bf2-45e3-b6ec-49a1b7cb963b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:37.911590 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a107411b-2bf2-45e3-b6ec-49a1b7cb963b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:37.919914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a107411b-2bf2-45e3-b6ec-49a1b7cb963b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145947 Jun 10 13:30:37.920479 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1310/2224] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:37 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:38.937764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b04076b2-82e5-481a-8dd8-4da24379f9a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:38.938635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b04076b2-82e5-481a-8dd8-4da24379f9a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:38.945802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b04076b2-82e5-481a-8dd8-4da24379f9a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:38.946242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b04076b2-82e5-481a-8dd8-4da24379f9a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:38.946998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b04076b2-82e5-481a-8dd8-4da24379f9a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:38.947428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b04076b2-82e5-481a-8dd8-4da24379f9a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:39.080499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b04076b2-82e5-481a-8dd8-4da24379f9a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:39.089001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b04076b2-82e5-481a-8dd8-4da24379f9a4 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.155957 Jun 10 13:30:39.089672 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1311/2225] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:38 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:40.105827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7579d024-eeda-499d-a645-14d6b4ba89e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:40.106474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7579d024-eeda-499d-a645-14d6b4ba89e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:40.113226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7579d024-eeda-499d-a645-14d6b4ba89e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:40.113681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7579d024-eeda-499d-a645-14d6b4ba89e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:40.114450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7579d024-eeda-499d-a645-14d6b4ba89e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:40.114886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7579d024-eeda-499d-a645-14d6b4ba89e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:40.250737 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7579d024-eeda-499d-a645-14d6b4ba89e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:40.259419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7579d024-eeda-499d-a645-14d6b4ba89e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.157547 Jun 10 13:30:40.260011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1312/2226] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:40 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:41.277291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f8f0beb2-db2b-47ec-b9a8-904b38f42b32 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:41.277926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f8f0beb2-db2b-47ec-b9a8-904b38f42b32 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:41.285287 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8f0beb2-db2b-47ec-b9a8-904b38f42b32 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:41.285770 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8f0beb2-db2b-47ec-b9a8-904b38f42b32 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:41.286545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8f0beb2-db2b-47ec-b9a8-904b38f42b32 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:41.286986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8f0beb2-db2b-47ec-b9a8-904b38f42b32 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:41.421227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f8f0beb2-db2b-47ec-b9a8-904b38f42b32 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:41.430044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f8f0beb2-db2b-47ec-b9a8-904b38f42b32 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.156566 Jun 10 13:30:41.430624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1313/2227] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:41 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:42.447379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5fedc452-b78d-4eff-bb9f-a0c5d06d5d69 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:42.448037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5fedc452-b78d-4eff-bb9f-a0c5d06d5d69 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:42.455268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5fedc452-b78d-4eff-bb9f-a0c5d06d5d69 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:42.455723 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5fedc452-b78d-4eff-bb9f-a0c5d06d5d69 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:42.456491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5fedc452-b78d-4eff-bb9f-a0c5d06d5d69 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:42.456957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5fedc452-b78d-4eff-bb9f-a0c5d06d5d69 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:42.592198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5fedc452-b78d-4eff-bb9f-a0c5d06d5d69 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:42.600385 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5fedc452-b78d-4eff-bb9f-a0c5d06d5d69 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.156931 Jun 10 13:30:42.600994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1314/2228] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:42 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:43.617659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-de292227-49af-4ac7-aa2a-d56c85dfb8a6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:43.618279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-de292227-49af-4ac7-aa2a-d56c85dfb8a6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:43.624941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-de292227-49af-4ac7-aa2a-d56c85dfb8a6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:43.625399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-de292227-49af-4ac7-aa2a-d56c85dfb8a6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:43.626167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-de292227-49af-4ac7-aa2a-d56c85dfb8a6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:43.626604 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-de292227-49af-4ac7-aa2a-d56c85dfb8a6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:43.758902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-de292227-49af-4ac7-aa2a-d56c85dfb8a6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:43.767663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-de292227-49af-4ac7-aa2a-d56c85dfb8a6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.153912 Jun 10 13:30:43.768317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1315/2229] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:43 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:44.784851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8d7f50af-de1a-4955-83c7-20bbb8e45da2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:44.785504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8d7f50af-de1a-4955-83c7-20bbb8e45da2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:44.792371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8d7f50af-de1a-4955-83c7-20bbb8e45da2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:44.792955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8d7f50af-de1a-4955-83c7-20bbb8e45da2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:44.793739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8d7f50af-de1a-4955-83c7-20bbb8e45da2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:44.794175 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8d7f50af-de1a-4955-83c7-20bbb8e45da2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:44.922665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8d7f50af-de1a-4955-83c7-20bbb8e45da2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:44.930640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8d7f50af-de1a-4955-83c7-20bbb8e45da2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149744 Jun 10 13:30:44.931192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1316/2230] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:44 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:45.948656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-331db5bb-4ec2-4261-9bb8-65137304d5e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:45.949340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-331db5bb-4ec2-4261-9bb8-65137304d5e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:45.955742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-331db5bb-4ec2-4261-9bb8-65137304d5e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:45.956208 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-331db5bb-4ec2-4261-9bb8-65137304d5e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:45.956978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-331db5bb-4ec2-4261-9bb8-65137304d5e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:45.957414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-331db5bb-4ec2-4261-9bb8-65137304d5e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:46.081671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-331db5bb-4ec2-4261-9bb8-65137304d5e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:46.089772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-331db5bb-4ec2-4261-9bb8-65137304d5e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145040 Jun 10 13:30:46.090334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1317/2231] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:45 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:47.106649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-653cc807-da06-469c-972c-646c177dcab1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:47.107323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-653cc807-da06-469c-972c-646c177dcab1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:47.113788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-653cc807-da06-469c-972c-646c177dcab1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:47.114240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-653cc807-da06-469c-972c-646c177dcab1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:47.115001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-653cc807-da06-469c-972c-646c177dcab1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:47.115434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-653cc807-da06-469c-972c-646c177dcab1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:47.244046 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-653cc807-da06-469c-972c-646c177dcab1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:47.252445 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-653cc807-da06-469c-972c-646c177dcab1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149679 Jun 10 13:30:47.253042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1318/2232] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:47 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:48.269092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bab4e85e-ee1c-40b5-a164-510807531e38 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:48.269779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bab4e85e-ee1c-40b5-a164-510807531e38 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:48.276211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bab4e85e-ee1c-40b5-a164-510807531e38 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:48.276662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bab4e85e-ee1c-40b5-a164-510807531e38 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:48.277450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bab4e85e-ee1c-40b5-a164-510807531e38 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:48.277887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bab4e85e-ee1c-40b5-a164-510807531e38 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:48.402095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bab4e85e-ee1c-40b5-a164-510807531e38 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:48.410677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bab4e85e-ee1c-40b5-a164-510807531e38 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145474 Jun 10 13:30:48.411244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1319/2233] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:48 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:49.427643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3f7ea4f9-b1c1-48b0-a5f2-fe7a80223d09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:49.428334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3f7ea4f9-b1c1-48b0-a5f2-fe7a80223d09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:49.434886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f7ea4f9-b1c1-48b0-a5f2-fe7a80223d09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:49.435331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f7ea4f9-b1c1-48b0-a5f2-fe7a80223d09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:49.436104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f7ea4f9-b1c1-48b0-a5f2-fe7a80223d09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:49.436517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3f7ea4f9-b1c1-48b0-a5f2-fe7a80223d09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:49.563443 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3f7ea4f9-b1c1-48b0-a5f2-fe7a80223d09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:49.571488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3f7ea4f9-b1c1-48b0-a5f2-fe7a80223d09 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147690 Jun 10 13:30:49.572058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1320/2234] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:49 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:50.588314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c37f7e90-302f-410e-9f1a-3627adf790c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:50.588955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c37f7e90-302f-410e-9f1a-3627adf790c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:50.595572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c37f7e90-302f-410e-9f1a-3627adf790c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:50.596013 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c37f7e90-302f-410e-9f1a-3627adf790c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:50.596774 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c37f7e90-302f-410e-9f1a-3627adf790c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:50.597218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c37f7e90-302f-410e-9f1a-3627adf790c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:50.723336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c37f7e90-302f-410e-9f1a-3627adf790c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:50.731651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c37f7e90-302f-410e-9f1a-3627adf790c5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146796 Jun 10 13:30:50.732219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1321/2235] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:50 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:51.750781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-04f86f02-8056-4fa9-9a26-3657c2728f34 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:51.751423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-04f86f02-8056-4fa9-9a26-3657c2728f34 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:51.757970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04f86f02-8056-4fa9-9a26-3657c2728f34 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:51.758449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04f86f02-8056-4fa9-9a26-3657c2728f34 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:51.759210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04f86f02-8056-4fa9-9a26-3657c2728f34 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:51.759649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-04f86f02-8056-4fa9-9a26-3657c2728f34 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:51.777663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:30:51.886644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-04f86f02-8056-4fa9-9a26-3657c2728f34 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:51.895794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-04f86f02-8056-4fa9-9a26-3657c2728f34 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149561 Jun 10 13:30:51.896365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1322/2236] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:51 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:52.913091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-acf0c1d0-6219-4804-ae7c-3a3757d6af1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:52.913730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-acf0c1d0-6219-4804-ae7c-3a3757d6af1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:52.920094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acf0c1d0-6219-4804-ae7c-3a3757d6af1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:52.920545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acf0c1d0-6219-4804-ae7c-3a3757d6af1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:52.921344 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acf0c1d0-6219-4804-ae7c-3a3757d6af1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:52.921777 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acf0c1d0-6219-4804-ae7c-3a3757d6af1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:53.050817 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-acf0c1d0-6219-4804-ae7c-3a3757d6af1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:53.058785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-acf0c1d0-6219-4804-ae7c-3a3757d6af1c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149689 Jun 10 13:30:53.059963 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1323/2237] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:52 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:54.076311 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b43f96d3-546f-4713-9bac-1967bae61ac7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:54.077020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b43f96d3-546f-4713-9bac-1967bae61ac7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:54.083618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b43f96d3-546f-4713-9bac-1967bae61ac7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:54.084055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b43f96d3-546f-4713-9bac-1967bae61ac7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:54.084823 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b43f96d3-546f-4713-9bac-1967bae61ac7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:54.085255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b43f96d3-546f-4713-9bac-1967bae61ac7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:54.216013 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b43f96d3-546f-4713-9bac-1967bae61ac7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:54.223796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b43f96d3-546f-4713-9bac-1967bae61ac7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.151291 Jun 10 13:30:54.224352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1324/2238] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:54 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:55.242577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-427a0ab9-c515-44b1-9114-3a7c66de0e62 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:55.243223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-427a0ab9-c515-44b1-9114-3a7c66de0e62 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:55.249716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-427a0ab9-c515-44b1-9114-3a7c66de0e62 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:55.250162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-427a0ab9-c515-44b1-9114-3a7c66de0e62 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:55.250912 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-427a0ab9-c515-44b1-9114-3a7c66de0e62 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:55.251340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-427a0ab9-c515-44b1-9114-3a7c66de0e62 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:55.379960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-427a0ab9-c515-44b1-9114-3a7c66de0e62 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:55.388433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-427a0ab9-c515-44b1-9114-3a7c66de0e62 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149648 Jun 10 13:30:55.389095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1325/2239] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:55 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:56.409619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-53d68aac-1dc2-4e9d-83f8-84ea1003a07d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:56.410227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-53d68aac-1dc2-4e9d-83f8-84ea1003a07d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:56.416670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53d68aac-1dc2-4e9d-83f8-84ea1003a07d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:56.417191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53d68aac-1dc2-4e9d-83f8-84ea1003a07d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:56.417939 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53d68aac-1dc2-4e9d-83f8-84ea1003a07d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:56.418936 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53d68aac-1dc2-4e9d-83f8-84ea1003a07d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:56.540395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-53d68aac-1dc2-4e9d-83f8-84ea1003a07d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:56.548218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-53d68aac-1dc2-4e9d-83f8-84ea1003a07d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.142474 Jun 10 13:30:56.548783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1326/2240] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:56 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:57.565091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5159138c-34f1-4e6d-805d-9b97c82695cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:57.565680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5159138c-34f1-4e6d-805d-9b97c82695cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:57.567985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=52 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:30:57.572782 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5159138c-34f1-4e6d-805d-9b97c82695cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:57.573245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5159138c-34f1-4e6d-805d-9b97c82695cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:57.574007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5159138c-34f1-4e6d-805d-9b97c82695cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:57.574446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5159138c-34f1-4e6d-805d-9b97c82695cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:57.700496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5159138c-34f1-4e6d-805d-9b97c82695cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:57.708909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5159138c-34f1-4e6d-805d-9b97c82695cc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147671 Jun 10 13:30:57.709482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1327/2241] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:57 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:58.726382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7d434a7d-1621-45ae-836e-37fdc5480806 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:58.726985 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7d434a7d-1621-45ae-836e-37fdc5480806 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:58.733459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d434a7d-1621-45ae-836e-37fdc5480806 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:58.733919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d434a7d-1621-45ae-836e-37fdc5480806 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:58.734682 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d434a7d-1621-45ae-836e-37fdc5480806 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:58.735108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7d434a7d-1621-45ae-836e-37fdc5480806 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:58.862110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7d434a7d-1621-45ae-836e-37fdc5480806 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:30:58.869959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7d434a7d-1621-45ae-836e-37fdc5480806 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147438 Jun 10 13:30:58.870527 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1328/2242] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:58 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:30:59.886493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-df01cdeb-9d93-4203-91ea-823941184626 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:30:59.887826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-df01cdeb-9d93-4203-91ea-823941184626 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:30:59.894167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df01cdeb-9d93-4203-91ea-823941184626 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:59.894612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df01cdeb-9d93-4203-91ea-823941184626 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:30:59.895366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df01cdeb-9d93-4203-91ea-823941184626 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:30:59.895798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df01cdeb-9d93-4203-91ea-823941184626 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:00.019931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-df01cdeb-9d93-4203-91ea-823941184626 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:00.027896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-df01cdeb-9d93-4203-91ea-823941184626 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145277 Jun 10 13:31:00.028462 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1329/2243] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:30:59 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:01.045686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6fe2d1fc-da2d-4004-9bb6-b45cdf8f7220 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:01.046354 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6fe2d1fc-da2d-4004-9bb6-b45cdf8f7220 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:01.052297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6fe2d1fc-da2d-4004-9bb6-b45cdf8f7220 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:01.052738 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6fe2d1fc-da2d-4004-9bb6-b45cdf8f7220 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:01.053510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6fe2d1fc-da2d-4004-9bb6-b45cdf8f7220 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:01.053946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6fe2d1fc-da2d-4004-9bb6-b45cdf8f7220 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:01.182643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6fe2d1fc-da2d-4004-9bb6-b45cdf8f7220 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:01.191185 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6fe2d1fc-da2d-4004-9bb6-b45cdf8f7220 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150045 Jun 10 13:31:01.191749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1330/2244] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:01 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:02.208162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8652eba1-0c25-4da3-85e6-7727c6224622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:02.208849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8652eba1-0c25-4da3-85e6-7727c6224622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:02.215226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8652eba1-0c25-4da3-85e6-7727c6224622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:02.215663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8652eba1-0c25-4da3-85e6-7727c6224622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:02.216416 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8652eba1-0c25-4da3-85e6-7727c6224622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:02.216878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8652eba1-0c25-4da3-85e6-7727c6224622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:02.343741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8652eba1-0c25-4da3-85e6-7727c6224622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:02.351626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8652eba1-0c25-4da3-85e6-7727c6224622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147585 Jun 10 13:31:02.352186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1331/2245] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:02 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:03.368831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-03c689d0-d87a-4ad3-90e7-3905fb0ca462 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:03.369479 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-03c689d0-d87a-4ad3-90e7-3905fb0ca462 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:03.375915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-03c689d0-d87a-4ad3-90e7-3905fb0ca462 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:03.376395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-03c689d0-d87a-4ad3-90e7-3905fb0ca462 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:03.377188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-03c689d0-d87a-4ad3-90e7-3905fb0ca462 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:03.377624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-03c689d0-d87a-4ad3-90e7-3905fb0ca462 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:03.511279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-03c689d0-d87a-4ad3-90e7-3905fb0ca462 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:03.519740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-03c689d0-d87a-4ad3-90e7-3905fb0ca462 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.154795 Jun 10 13:31:03.520503 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1332/2246] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:03 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:04.537983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c7ee765c-618f-4ee6-a98d-6037bdc3ea68 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:04.538643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c7ee765c-618f-4ee6-a98d-6037bdc3ea68 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:04.545179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7ee765c-618f-4ee6-a98d-6037bdc3ea68 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:04.545627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7ee765c-618f-4ee6-a98d-6037bdc3ea68 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:04.546383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7ee765c-618f-4ee6-a98d-6037bdc3ea68 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:04.546821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7ee765c-618f-4ee6-a98d-6037bdc3ea68 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:04.676627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c7ee765c-618f-4ee6-a98d-6037bdc3ea68 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:04.677962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c7ee765c-618f-4ee6-a98d-6037bdc3ea68 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143826 Jun 10 13:31:04.678526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1333/2247] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:04 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:05.695384 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-06e5937d-c2d0-492f-bb29-18b32b53391b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:05.696039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-06e5937d-c2d0-492f-bb29-18b32b53391b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:05.702720 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06e5937d-c2d0-492f-bb29-18b32b53391b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:05.703194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06e5937d-c2d0-492f-bb29-18b32b53391b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:05.703942 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06e5937d-c2d0-492f-bb29-18b32b53391b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:05.704370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-06e5937d-c2d0-492f-bb29-18b32b53391b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:05.890287 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-06e5937d-c2d0-492f-bb29-18b32b53391b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:05.898175 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-06e5937d-c2d0-492f-bb29-18b32b53391b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.206659 Jun 10 13:31:05.898730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1334/2248] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:05 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:06.915020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-13dd0a45-4c36-45df-9061-b13800e5ef5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:06.915647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-13dd0a45-4c36-45df-9061-b13800e5ef5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:06.922476 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-13dd0a45-4c36-45df-9061-b13800e5ef5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:06.926195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-13dd0a45-4c36-45df-9061-b13800e5ef5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:06.926195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-13dd0a45-4c36-45df-9061-b13800e5ef5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:06.926195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-13dd0a45-4c36-45df-9061-b13800e5ef5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:07.052339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-13dd0a45-4c36-45df-9061-b13800e5ef5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:07.060892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-13dd0a45-4c36-45df-9061-b13800e5ef5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149863 Jun 10 13:31:07.061465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1335/2249] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:06 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:08.077782 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e5a8f115-ccc0-440f-a006-060505875206 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:08.078422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e5a8f115-ccc0-440f-a006-060505875206 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:08.085237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e5a8f115-ccc0-440f-a006-060505875206 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:08.085685 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e5a8f115-ccc0-440f-a006-060505875206 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:08.086440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e5a8f115-ccc0-440f-a006-060505875206 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:08.086878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e5a8f115-ccc0-440f-a006-060505875206 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:08.211248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e5a8f115-ccc0-440f-a006-060505875206 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:08.219595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e5a8f115-ccc0-440f-a006-060505875206 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145717 Jun 10 13:31:08.220159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1336/2250] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:08 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:09.236973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f314e5ba-4afd-49bb-9ad0-fb080e074bc9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:09.237720 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f314e5ba-4afd-49bb-9ad0-fb080e074bc9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:09.244063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f314e5ba-4afd-49bb-9ad0-fb080e074bc9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:09.244515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f314e5ba-4afd-49bb-9ad0-fb080e074bc9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:09.245294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f314e5ba-4afd-49bb-9ad0-fb080e074bc9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:09.245728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f314e5ba-4afd-49bb-9ad0-fb080e074bc9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:09.375930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f314e5ba-4afd-49bb-9ad0-fb080e074bc9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:09.383899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f314e5ba-4afd-49bb-9ad0-fb080e074bc9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150844 Jun 10 13:31:09.384460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1337/2251] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:09 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:10.401537 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-db6c77ac-ac35-4445-991e-5d33879b9cde tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:10.402650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-db6c77ac-ac35-4445-991e-5d33879b9cde tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:10.411291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-db6c77ac-ac35-4445-991e-5d33879b9cde tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:10.411739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-db6c77ac-ac35-4445-991e-5d33879b9cde tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:10.412492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-db6c77ac-ac35-4445-991e-5d33879b9cde tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:10.412954 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-db6c77ac-ac35-4445-991e-5d33879b9cde tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:10.537988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-db6c77ac-ac35-4445-991e-5d33879b9cde tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:10.545824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-db6c77ac-ac35-4445-991e-5d33879b9cde tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148717 Jun 10 13:31:10.546390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1338/2252] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:10 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:31:11.561855 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3bd8cbc3-7595-40e1-a3e1-fa144202c8e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:11.562459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3bd8cbc3-7595-40e1-a3e1-fa144202c8e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:11.569122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3bd8cbc3-7595-40e1-a3e1-fa144202c8e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:11.569576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3bd8cbc3-7595-40e1-a3e1-fa144202c8e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:11.570367 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3bd8cbc3-7595-40e1-a3e1-fa144202c8e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:11.570807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3bd8cbc3-7595-40e1-a3e1-fa144202c8e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:11.575264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:31:11.697636 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3bd8cbc3-7595-40e1-a3e1-fa144202c8e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:11.706214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3bd8cbc3-7595-40e1-a3e1-fa144202c8e0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148196 Jun 10 13:31:11.706779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1339/2253] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:11 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:12.724089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-25684b9c-e347-4223-be0a-b49c28f4be6b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:12.724687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-25684b9c-e347-4223-be0a-b49c28f4be6b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:12.731809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25684b9c-e347-4223-be0a-b49c28f4be6b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:12.732254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25684b9c-e347-4223-be0a-b49c28f4be6b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:12.733040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25684b9c-e347-4223-be0a-b49c28f4be6b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:12.733477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-25684b9c-e347-4223-be0a-b49c28f4be6b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:12.856277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-25684b9c-e347-4223-be0a-b49c28f4be6b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:12.864095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-25684b9c-e347-4223-be0a-b49c28f4be6b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143857 Jun 10 13:31:12.864657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1340/2254] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:12 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:13.881188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5c5edca4-600d-4162-a28c-faf4097c0d3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:13.881977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5c5edca4-600d-4162-a28c-faf4097c0d3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:13.888492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c5edca4-600d-4162-a28c-faf4097c0d3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:13.888965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c5edca4-600d-4162-a28c-faf4097c0d3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:13.889714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c5edca4-600d-4162-a28c-faf4097c0d3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:13.890139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5c5edca4-600d-4162-a28c-faf4097c0d3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:14.013226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5c5edca4-600d-4162-a28c-faf4097c0d3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:14.020970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5c5edca4-600d-4162-a28c-faf4097c0d3d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143599 Jun 10 13:31:14.021562 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1341/2255] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:13 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:15.038388 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1bb725fd-0a9b-452f-9e54-ddec0bf53d42 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:15.039034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1bb725fd-0a9b-452f-9e54-ddec0bf53d42 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:15.046153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1bb725fd-0a9b-452f-9e54-ddec0bf53d42 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:15.046595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1bb725fd-0a9b-452f-9e54-ddec0bf53d42 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:15.047341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1bb725fd-0a9b-452f-9e54-ddec0bf53d42 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:15.047766 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1bb725fd-0a9b-452f-9e54-ddec0bf53d42 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:15.359121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1bb725fd-0a9b-452f-9e54-ddec0bf53d42 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:15.366797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1bb725fd-0a9b-452f-9e54-ddec0bf53d42 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.332341 Jun 10 13:31:15.367357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1342/2256] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:15 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:16.384112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-58911dbf-cf6e-441e-a822-c58639d1acbe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:16.384724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-58911dbf-cf6e-441e-a822-c58639d1acbe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:16.392056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58911dbf-cf6e-441e-a822-c58639d1acbe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:16.392523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58911dbf-cf6e-441e-a822-c58639d1acbe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:16.393278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58911dbf-cf6e-441e-a822-c58639d1acbe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:16.393709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58911dbf-cf6e-441e-a822-c58639d1acbe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:16.517742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-58911dbf-cf6e-441e-a822-c58639d1acbe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:16.525722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-58911dbf-cf6e-441e-a822-c58639d1acbe tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145541 Jun 10 13:31:16.526286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1343/2257] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:16 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:17.542746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-67c648b5-aa0d-4a3a-9f18-a362b56e7059 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:17.543372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-67c648b5-aa0d-4a3a-9f18-a362b56e7059 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:17.549683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-67c648b5-aa0d-4a3a-9f18-a362b56e7059 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:17.550127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-67c648b5-aa0d-4a3a-9f18-a362b56e7059 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:17.550876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-67c648b5-aa0d-4a3a-9f18-a362b56e7059 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:17.551306 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-67c648b5-aa0d-4a3a-9f18-a362b56e7059 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:17.676466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-67c648b5-aa0d-4a3a-9f18-a362b56e7059 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:17.684976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-67c648b5-aa0d-4a3a-9f18-a362b56e7059 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146605 Jun 10 13:31:17.685548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1344/2258] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:17 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:18.702044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1ff41fd1-2d46-402d-8ee9-3bd071087c85 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:18.702681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1ff41fd1-2d46-402d-8ee9-3bd071087c85 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:18.709468 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ff41fd1-2d46-402d-8ee9-3bd071087c85 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:18.709917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ff41fd1-2d46-402d-8ee9-3bd071087c85 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:18.710670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ff41fd1-2d46-402d-8ee9-3bd071087c85 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:18.711102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ff41fd1-2d46-402d-8ee9-3bd071087c85 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:18.840895 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1ff41fd1-2d46-402d-8ee9-3bd071087c85 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:18.849215 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1ff41fd1-2d46-402d-8ee9-3bd071087c85 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.151540 Jun 10 13:31:18.849778 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1345/2259] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:18 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:19.866236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ce66f1d1-67c6-4dcb-a7c2-9f231d21a20a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:19.867018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ce66f1d1-67c6-4dcb-a7c2-9f231d21a20a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:19.874106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ce66f1d1-67c6-4dcb-a7c2-9f231d21a20a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:19.874556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ce66f1d1-67c6-4dcb-a7c2-9f231d21a20a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:19.875292 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ce66f1d1-67c6-4dcb-a7c2-9f231d21a20a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:19.875720 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ce66f1d1-67c6-4dcb-a7c2-9f231d21a20a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:20.002605 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ce66f1d1-67c6-4dcb-a7c2-9f231d21a20a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:20.011123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ce66f1d1-67c6-4dcb-a7c2-9f231d21a20a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148694 Jun 10 13:31:20.011700 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1346/2260] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:19 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:21.029109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0f577e38-12f4-4bcf-8b05-070750e47151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:21.029746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0f577e38-12f4-4bcf-8b05-070750e47151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:21.036214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f577e38-12f4-4bcf-8b05-070750e47151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:21.036666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f577e38-12f4-4bcf-8b05-070750e47151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:21.037460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f577e38-12f4-4bcf-8b05-070750e47151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:21.037897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f577e38-12f4-4bcf-8b05-070750e47151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:21.164891 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0f577e38-12f4-4bcf-8b05-070750e47151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:21.173180 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0f577e38-12f4-4bcf-8b05-070750e47151 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148029 Jun 10 13:31:21.173741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1347/2261] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:21 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:22.190134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a4e461f8-0388-4149-b318-af7cc0c4bbcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:22.190783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a4e461f8-0388-4149-b318-af7cc0c4bbcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:22.197226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a4e461f8-0388-4149-b318-af7cc0c4bbcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:22.197670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a4e461f8-0388-4149-b318-af7cc0c4bbcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:22.198581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a4e461f8-0388-4149-b318-af7cc0c4bbcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:22.199014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a4e461f8-0388-4149-b318-af7cc0c4bbcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:22.337552 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a4e461f8-0388-4149-b318-af7cc0c4bbcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:22.346152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a4e461f8-0388-4149-b318-af7cc0c4bbcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.160075 Jun 10 13:31:22.346716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1348/2262] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:22 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:23.363428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-63367b9f-a99a-469e-922a-ecd432baaa88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:23.364044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-63367b9f-a99a-469e-922a-ecd432baaa88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:23.371131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63367b9f-a99a-469e-922a-ecd432baaa88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:23.371579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63367b9f-a99a-469e-922a-ecd432baaa88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:23.372336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63367b9f-a99a-469e-922a-ecd432baaa88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:23.372797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63367b9f-a99a-469e-922a-ecd432baaa88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:23.504803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-63367b9f-a99a-469e-922a-ecd432baaa88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:23.512743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-63367b9f-a99a-469e-922a-ecd432baaa88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.153209 Jun 10 13:31:23.513326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1349/2263] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:23 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:24.382457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:31:24.383047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:31:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:31:24.530252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9fc467c7-0a52-4c7b-8b41-ffc06b73c2b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:24.530853 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9fc467c7-0a52-4c7b-8b41-ffc06b73c2b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:24.537285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9fc467c7-0a52-4c7b-8b41-ffc06b73c2b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:24.537729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9fc467c7-0a52-4c7b-8b41-ffc06b73c2b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:24.538476 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9fc467c7-0a52-4c7b-8b41-ffc06b73c2b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:24.538906 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9fc467c7-0a52-4c7b-8b41-ffc06b73c2b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:24.662498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9fc467c7-0a52-4c7b-8b41-ffc06b73c2b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:24.671238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9fc467c7-0a52-4c7b-8b41-ffc06b73c2b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145173 Jun 10 13:31:24.671674 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1350/2264] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:24 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:25.691021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-22863640-16ec-4445-bba7-3a154ae4dc53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:25.692109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-22863640-16ec-4445-bba7-3a154ae4dc53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:25.699768 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-22863640-16ec-4445-bba7-3a154ae4dc53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:25.700215 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-22863640-16ec-4445-bba7-3a154ae4dc53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:25.701011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-22863640-16ec-4445-bba7-3a154ae4dc53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:25.701448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-22863640-16ec-4445-bba7-3a154ae4dc53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:25.837800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-22863640-16ec-4445-bba7-3a154ae4dc53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:25.845704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-22863640-16ec-4445-bba7-3a154ae4dc53 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.161291 Jun 10 13:31:25.846265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1351/2265] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:25 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:26.863244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c288ed53-f072-4f6b-b196-75f9a4b74610 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:26.863905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c288ed53-f072-4f6b-b196-75f9a4b74610 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:26.870327 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c288ed53-f072-4f6b-b196-75f9a4b74610 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:26.870776 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c288ed53-f072-4f6b-b196-75f9a4b74610 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:26.871537 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c288ed53-f072-4f6b-b196-75f9a4b74610 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:26.871970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c288ed53-f072-4f6b-b196-75f9a4b74610 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:26.995656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c288ed53-f072-4f6b-b196-75f9a4b74610 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:27.003467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c288ed53-f072-4f6b-b196-75f9a4b74610 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144105 Jun 10 13:31:27.004028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1352/2266] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:26 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:28.020353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f14e19ba-0c26-444f-899f-8039981a3e25 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:28.021045 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f14e19ba-0c26-444f-899f-8039981a3e25 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:28.023306 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=52 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:31:28.028584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f14e19ba-0c26-444f-899f-8039981a3e25 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:28.029072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f14e19ba-0c26-444f-899f-8039981a3e25 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:28.029826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f14e19ba-0c26-444f-899f-8039981a3e25 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:28.030263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f14e19ba-0c26-444f-899f-8039981a3e25 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:28.158532 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f14e19ba-0c26-444f-899f-8039981a3e25 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:28.166455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f14e19ba-0c26-444f-899f-8039981a3e25 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149988 Jun 10 13:31:28.167011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1353/2267] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:28 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:29.183295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-80cc4262-a5f3-4f42-89b6-3a658ff017f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:29.183968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-80cc4262-a5f3-4f42-89b6-3a658ff017f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:29.190356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80cc4262-a5f3-4f42-89b6-3a658ff017f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:29.190801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80cc4262-a5f3-4f42-89b6-3a658ff017f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:29.191543 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80cc4262-a5f3-4f42-89b6-3a658ff017f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:29.191971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80cc4262-a5f3-4f42-89b6-3a658ff017f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:29.318073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-80cc4262-a5f3-4f42-89b6-3a658ff017f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:29.326766 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-80cc4262-a5f3-4f42-89b6-3a658ff017f0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147278 Jun 10 13:31:29.327341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1354/2268] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:29 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:30.344548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-31081229-0977-40ba-8d70-03c14079c9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:30.345179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-31081229-0977-40ba-8d70-03c14079c9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:30.351971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31081229-0977-40ba-8d70-03c14079c9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:30.352411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31081229-0977-40ba-8d70-03c14079c9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:30.353187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31081229-0977-40ba-8d70-03c14079c9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:30.353612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31081229-0977-40ba-8d70-03c14079c9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:30.481196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-31081229-0977-40ba-8d70-03c14079c9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:30.484541 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:31:30.489283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-31081229-0977-40ba-8d70-03c14079c9f7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148583 Jun 10 13:31:30.489843 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1355/2269] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:30 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:31.506693 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-67ec4efa-2275-4c45-ba3f-68fb4fbaccf6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:31.507394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-67ec4efa-2275-4c45-ba3f-68fb4fbaccf6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:31.513991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-67ec4efa-2275-4c45-ba3f-68fb4fbaccf6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:31.514433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-67ec4efa-2275-4c45-ba3f-68fb4fbaccf6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:31.515177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-67ec4efa-2275-4c45-ba3f-68fb4fbaccf6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:31.515604 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-67ec4efa-2275-4c45-ba3f-68fb4fbaccf6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:31.641254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-67ec4efa-2275-4c45-ba3f-68fb4fbaccf6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:31.649011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-67ec4efa-2275-4c45-ba3f-68fb4fbaccf6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146621 Jun 10 13:31:31.649567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1356/2270] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:31 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:32.665729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-578a2485-dde7-4b85-9b95-30013a4001e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:32.666346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-578a2485-dde7-4b85-9b95-30013a4001e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:32.673173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-578a2485-dde7-4b85-9b95-30013a4001e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:32.673611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-578a2485-dde7-4b85-9b95-30013a4001e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:32.674382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-578a2485-dde7-4b85-9b95-30013a4001e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:32.674812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-578a2485-dde7-4b85-9b95-30013a4001e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:32.797600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-578a2485-dde7-4b85-9b95-30013a4001e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:32.805340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-578a2485-dde7-4b85-9b95-30013a4001e5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143425 Jun 10 13:31:32.805890 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1357/2271] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:32 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:33.822769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b232b5a1-65d5-45eb-862b-25a753d88051 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:33.823454 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b232b5a1-65d5-45eb-862b-25a753d88051 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:33.829851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b232b5a1-65d5-45eb-862b-25a753d88051 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:33.830295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b232b5a1-65d5-45eb-862b-25a753d88051 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:33.831045 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b232b5a1-65d5-45eb-862b-25a753d88051 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:33.831729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b232b5a1-65d5-45eb-862b-25a753d88051 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:33.958705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b232b5a1-65d5-45eb-862b-25a753d88051 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:33.967063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b232b5a1-65d5-45eb-862b-25a753d88051 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148118 Jun 10 13:31:33.967620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1358/2272] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:33 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:34.983749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3c87d304-43be-44ee-b034-83d9ecbfb20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:34.984408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3c87d304-43be-44ee-b034-83d9ecbfb20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:34.990835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c87d304-43be-44ee-b034-83d9ecbfb20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:34.991277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c87d304-43be-44ee-b034-83d9ecbfb20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:34.992028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c87d304-43be-44ee-b034-83d9ecbfb20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:34.992453 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3c87d304-43be-44ee-b034-83d9ecbfb20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:35.124003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3c87d304-43be-44ee-b034-83d9ecbfb20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:35.131942 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3c87d304-43be-44ee-b034-83d9ecbfb20e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.151969 Jun 10 13:31:35.132501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1359/2273] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:34 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:36.149501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4a91ed31-8c3c-4675-a8b9-08f157977c13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:36.150189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4a91ed31-8c3c-4675-a8b9-08f157977c13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:36.157176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a91ed31-8c3c-4675-a8b9-08f157977c13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:36.157620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a91ed31-8c3c-4675-a8b9-08f157977c13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:36.158364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a91ed31-8c3c-4675-a8b9-08f157977c13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:36.158789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4a91ed31-8c3c-4675-a8b9-08f157977c13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:36.283774 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4a91ed31-8c3c-4675-a8b9-08f157977c13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:36.291621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4a91ed31-8c3c-4675-a8b9-08f157977c13 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145900 Jun 10 13:31:36.292173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1360/2274] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:36 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:37.308412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-678f61e0-f686-423d-90f8-8061f96b0bea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:37.309145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-678f61e0-f686-423d-90f8-8061f96b0bea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:37.315583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-678f61e0-f686-423d-90f8-8061f96b0bea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:37.316028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-678f61e0-f686-423d-90f8-8061f96b0bea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:37.316788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-678f61e0-f686-423d-90f8-8061f96b0bea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:37.317220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-678f61e0-f686-423d-90f8-8061f96b0bea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:37.446736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-678f61e0-f686-423d-90f8-8061f96b0bea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:37.454757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-678f61e0-f686-423d-90f8-8061f96b0bea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150254 Jun 10 13:31:37.455324 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1361/2275] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:37 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:38.472775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2deb577a-b747-47d4-a895-5390108e2acf tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:38.473420 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2deb577a-b747-47d4-a895-5390108e2acf tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:38.480434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2deb577a-b747-47d4-a895-5390108e2acf tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:38.480909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2deb577a-b747-47d4-a895-5390108e2acf tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:38.481661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2deb577a-b747-47d4-a895-5390108e2acf tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:38.482093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2deb577a-b747-47d4-a895-5390108e2acf tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:38.607618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2deb577a-b747-47d4-a895-5390108e2acf tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:38.615533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2deb577a-b747-47d4-a895-5390108e2acf tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146610 Jun 10 13:31:38.616058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1362/2276] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:38 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:39.632987 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a652453f-ed62-4c4b-a84f-e1d6e7c3abdd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:39.633638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a652453f-ed62-4c4b-a84f-e1d6e7c3abdd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:39.640063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a652453f-ed62-4c4b-a84f-e1d6e7c3abdd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:39.640538 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a652453f-ed62-4c4b-a84f-e1d6e7c3abdd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:39.641319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a652453f-ed62-4c4b-a84f-e1d6e7c3abdd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:39.641752 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a652453f-ed62-4c4b-a84f-e1d6e7c3abdd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:39.768794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a652453f-ed62-4c4b-a84f-e1d6e7c3abdd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:39.776875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a652453f-ed62-4c4b-a84f-e1d6e7c3abdd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147759 Jun 10 13:31:39.777488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1363/2277] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:39 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:40.793958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1ba2e1f8-eac6-47b9-b4b6-2bec01172b4a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:40.794635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1ba2e1f8-eac6-47b9-b4b6-2bec01172b4a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:40.801581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ba2e1f8-eac6-47b9-b4b6-2bec01172b4a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:40.802026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ba2e1f8-eac6-47b9-b4b6-2bec01172b4a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:40.802968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ba2e1f8-eac6-47b9-b4b6-2bec01172b4a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:40.803401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ba2e1f8-eac6-47b9-b4b6-2bec01172b4a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:40.929003 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1ba2e1f8-eac6-47b9-b4b6-2bec01172b4a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:40.936876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1ba2e1f8-eac6-47b9-b4b6-2bec01172b4a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146754 Jun 10 13:31:40.937441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1364/2278] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:40 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:41.954399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-76c5c40a-ad59-49d8-bbdc-91054f994dcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:41.955090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-76c5c40a-ad59-49d8-bbdc-91054f994dcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:41.961744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76c5c40a-ad59-49d8-bbdc-91054f994dcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:41.962192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76c5c40a-ad59-49d8-bbdc-91054f994dcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:41.962975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76c5c40a-ad59-49d8-bbdc-91054f994dcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:41.963407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-76c5c40a-ad59-49d8-bbdc-91054f994dcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:42.087340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-76c5c40a-ad59-49d8-bbdc-91054f994dcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:42.095637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-76c5c40a-ad59-49d8-bbdc-91054f994dcd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145109 Jun 10 13:31:42.096197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1365/2279] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:41 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:43.112550 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ae6ccc73-8c19-4726-b78f-1d6927b0b5f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:43.113246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ae6ccc73-8c19-4726-b78f-1d6927b0b5f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:43.120401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae6ccc73-8c19-4726-b78f-1d6927b0b5f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:43.120878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae6ccc73-8c19-4726-b78f-1d6927b0b5f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:43.121636 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae6ccc73-8c19-4726-b78f-1d6927b0b5f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:43.122072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ae6ccc73-8c19-4726-b78f-1d6927b0b5f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:43.250144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ae6ccc73-8c19-4726-b78f-1d6927b0b5f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:43.257995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ae6ccc73-8c19-4726-b78f-1d6927b0b5f2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149293 Jun 10 13:31:43.258550 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1366/2280] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:43 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:44.274768 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a5d7acac-446e-4bb0-a42e-ae5a5ae9ae90 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:44.275465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a5d7acac-446e-4bb0-a42e-ae5a5ae9ae90 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:44.282376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a5d7acac-446e-4bb0-a42e-ae5a5ae9ae90 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:44.282820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a5d7acac-446e-4bb0-a42e-ae5a5ae9ae90 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:44.283569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a5d7acac-446e-4bb0-a42e-ae5a5ae9ae90 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:44.283999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a5d7acac-446e-4bb0-a42e-ae5a5ae9ae90 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:44.410345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a5d7acac-446e-4bb0-a42e-ae5a5ae9ae90 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:44.419140 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a5d7acac-446e-4bb0-a42e-ae5a5ae9ae90 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148149 Jun 10 13:31:44.421763 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1367/2281] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:44 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:45.436358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a57861e7-9fe4-4bfd-8757-3dbc91cbaae9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:45.437001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a57861e7-9fe4-4bfd-8757-3dbc91cbaae9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:45.443601 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a57861e7-9fe4-4bfd-8757-3dbc91cbaae9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:45.444048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a57861e7-9fe4-4bfd-8757-3dbc91cbaae9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:45.444819 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a57861e7-9fe4-4bfd-8757-3dbc91cbaae9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:45.445251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a57861e7-9fe4-4bfd-8757-3dbc91cbaae9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:45.576259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a57861e7-9fe4-4bfd-8757-3dbc91cbaae9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:45.584303 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a57861e7-9fe4-4bfd-8757-3dbc91cbaae9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.151784 Jun 10 13:31:45.584882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1368/2282] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:45 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:46.601193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f1e80826-62f0-4d19-aecf-d80f3fd78e26 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:46.601825 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f1e80826-62f0-4d19-aecf-d80f3fd78e26 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:46.608171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1e80826-62f0-4d19-aecf-d80f3fd78e26 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:46.608617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1e80826-62f0-4d19-aecf-d80f3fd78e26 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:46.609398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1e80826-62f0-4d19-aecf-d80f3fd78e26 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:46.609837 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1e80826-62f0-4d19-aecf-d80f3fd78e26 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:46.735080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f1e80826-62f0-4d19-aecf-d80f3fd78e26 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:46.742946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f1e80826-62f0-4d19-aecf-d80f3fd78e26 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.145687 Jun 10 13:31:46.743510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1369/2283] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:46 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:47.760787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-34595413-6978-4a2c-bc50-bdfa77b6fb22 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:47.761408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-34595413-6978-4a2c-bc50-bdfa77b6fb22 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:47.767913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-34595413-6978-4a2c-bc50-bdfa77b6fb22 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:47.768362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-34595413-6978-4a2c-bc50-bdfa77b6fb22 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:47.769157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-34595413-6978-4a2c-bc50-bdfa77b6fb22 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:47.769592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-34595413-6978-4a2c-bc50-bdfa77b6fb22 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:47.896599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-34595413-6978-4a2c-bc50-bdfa77b6fb22 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:47.904549 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-34595413-6978-4a2c-bc50-bdfa77b6fb22 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148149 Jun 10 13:31:47.905152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1370/2284] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:47 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:48.921704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-964896f7-c374-4869-af95-200dc2e1450c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:48.922382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-964896f7-c374-4869-af95-200dc2e1450c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:48.929465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-964896f7-c374-4869-af95-200dc2e1450c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:48.929906 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-964896f7-c374-4869-af95-200dc2e1450c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:48.930655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-964896f7-c374-4869-af95-200dc2e1450c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:48.931082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-964896f7-c374-4869-af95-200dc2e1450c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:49.057234 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-964896f7-c374-4869-af95-200dc2e1450c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:49.065162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-964896f7-c374-4869-af95-200dc2e1450c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147410 Jun 10 13:31:49.065726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1371/2285] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:48 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:50.083419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-68ded8cb-0473-4664-a44b-0152445600ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:50.084173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-68ded8cb-0473-4664-a44b-0152445600ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:50.092316 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68ded8cb-0473-4664-a44b-0152445600ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:50.093056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68ded8cb-0473-4664-a44b-0152445600ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:50.094176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68ded8cb-0473-4664-a44b-0152445600ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:50.094871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68ded8cb-0473-4664-a44b-0152445600ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:50.115759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:31:50.236911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-68ded8cb-0473-4664-a44b-0152445600ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:50.247211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-68ded8cb-0473-4664-a44b-0152445600ea tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.167885 Jun 10 13:31:50.247927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1372/2286] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:50 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:51.264834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a5106a64-c4e4-4a81-8c03-0b1d17e02f30 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:51.265537 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a5106a64-c4e4-4a81-8c03-0b1d17e02f30 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:51.271952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a5106a64-c4e4-4a81-8c03-0b1d17e02f30 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:51.272401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a5106a64-c4e4-4a81-8c03-0b1d17e02f30 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:51.273207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a5106a64-c4e4-4a81-8c03-0b1d17e02f30 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:51.273645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a5106a64-c4e4-4a81-8c03-0b1d17e02f30 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:51.403649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a5106a64-c4e4-4a81-8c03-0b1d17e02f30 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:51.411444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a5106a64-c4e4-4a81-8c03-0b1d17e02f30 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150521 Jun 10 13:31:51.412008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1373/2287] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:51 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:52.428989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ca28c6a7-dc32-4088-91ba-82596bc362f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:52.429671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ca28c6a7-dc32-4088-91ba-82596bc362f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:52.436077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca28c6a7-dc32-4088-91ba-82596bc362f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:52.436530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca28c6a7-dc32-4088-91ba-82596bc362f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:52.437322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca28c6a7-dc32-4088-91ba-82596bc362f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:52.437761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca28c6a7-dc32-4088-91ba-82596bc362f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:52.562001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ca28c6a7-dc32-4088-91ba-82596bc362f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:52.569916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ca28c6a7-dc32-4088-91ba-82596bc362f1 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144861 Jun 10 13:31:52.570479 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1374/2288] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:52 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:53.587118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c624b666-af13-4481-9aa8-0ee0dc562752 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:53.587702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c624b666-af13-4481-9aa8-0ee0dc562752 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:53.594290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c624b666-af13-4481-9aa8-0ee0dc562752 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:53.594731 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c624b666-af13-4481-9aa8-0ee0dc562752 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:53.595475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c624b666-af13-4481-9aa8-0ee0dc562752 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:53.595902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c624b666-af13-4481-9aa8-0ee0dc562752 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:53.724375 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c624b666-af13-4481-9aa8-0ee0dc562752 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:53.732948 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c624b666-af13-4481-9aa8-0ee0dc562752 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149669 Jun 10 13:31:53.733520 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1375/2289] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:53 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:54.750057 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-26e9df45-ee60-41e2-a01c-e3854ef7a02a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:54.750705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-26e9df45-ee60-41e2-a01c-e3854ef7a02a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:54.757175 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26e9df45-ee60-41e2-a01c-e3854ef7a02a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:54.757615 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26e9df45-ee60-41e2-a01c-e3854ef7a02a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:54.758360 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26e9df45-ee60-41e2-a01c-e3854ef7a02a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:54.758787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26e9df45-ee60-41e2-a01c-e3854ef7a02a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:54.884038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-26e9df45-ee60-41e2-a01c-e3854ef7a02a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:54.892448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-26e9df45-ee60-41e2-a01c-e3854ef7a02a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146253 Jun 10 13:31:54.893051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1376/2290] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:54 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:55.909578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-476ed29b-2f97-4c55-a9a7-6126d40bc6ed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:55.910228 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-476ed29b-2f97-4c55-a9a7-6126d40bc6ed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:55.917164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-476ed29b-2f97-4c55-a9a7-6126d40bc6ed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:55.917608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-476ed29b-2f97-4c55-a9a7-6126d40bc6ed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:55.918352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-476ed29b-2f97-4c55-a9a7-6126d40bc6ed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:55.918782 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-476ed29b-2f97-4c55-a9a7-6126d40bc6ed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:56.045188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-476ed29b-2f97-4c55-a9a7-6126d40bc6ed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:56.053130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-476ed29b-2f97-4c55-a9a7-6126d40bc6ed tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147381 Jun 10 13:31:56.053698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1377/2291] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:55 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:57.070035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c6096824-a91a-4686-a626-7a99fb722176 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:57.070726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c6096824-a91a-4686-a626-7a99fb722176 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:57.077749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6096824-a91a-4686-a626-7a99fb722176 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:57.078197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6096824-a91a-4686-a626-7a99fb722176 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:57.078945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6096824-a91a-4686-a626-7a99fb722176 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:57.079375 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6096824-a91a-4686-a626-7a99fb722176 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:57.205475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c6096824-a91a-4686-a626-7a99fb722176 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:57.213731 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c6096824-a91a-4686-a626-7a99fb722176 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147594 Jun 10 13:31:57.214303 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1378/2292] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:57 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:58.230579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-63aa7122-4d3f-4e95-8cf2-b0574547447c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:58.231248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-63aa7122-4d3f-4e95-8cf2-b0574547447c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:58.233507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=52 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:31:58.238600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63aa7122-4d3f-4e95-8cf2-b0574547447c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:58.239052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63aa7122-4d3f-4e95-8cf2-b0574547447c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:58.239844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63aa7122-4d3f-4e95-8cf2-b0574547447c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:58.240278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63aa7122-4d3f-4e95-8cf2-b0574547447c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:58.367403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-63aa7122-4d3f-4e95-8cf2-b0574547447c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:58.375761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-63aa7122-4d3f-4e95-8cf2-b0574547447c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149012 Jun 10 13:31:58.376321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1379/2293] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:58 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:31:59.393230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-880324d4-c0ea-4ce2-9b43-8615163456e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:31:59.393835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-880324d4-c0ea-4ce2-9b43-8615163456e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:31:59.400369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-880324d4-c0ea-4ce2-9b43-8615163456e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:59.400895 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-880324d4-c0ea-4ce2-9b43-8615163456e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:59.401651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-880324d4-c0ea-4ce2-9b43-8615163456e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:31:59.402079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-880324d4-c0ea-4ce2-9b43-8615163456e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:31:59.529362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-880324d4-c0ea-4ce2-9b43-8615163456e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:31:59.537119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-880324d4-c0ea-4ce2-9b43-8615163456e7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.147666 Jun 10 13:31:59.537688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1380/2294] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:31:59 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:00.555277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6c77d90c-0044-4172-85fb-95a44bd63c79 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:00.555929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6c77d90c-0044-4172-85fb-95a44bd63c79 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:00.562896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c77d90c-0044-4172-85fb-95a44bd63c79 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:00.563344 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c77d90c-0044-4172-85fb-95a44bd63c79 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:00.564094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c77d90c-0044-4172-85fb-95a44bd63c79 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:00.564529 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6c77d90c-0044-4172-85fb-95a44bd63c79 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:00.691503 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6c77d90c-0044-4172-85fb-95a44bd63c79 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:00.699988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6c77d90c-0044-4172-85fb-95a44bd63c79 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148533 Jun 10 13:32:00.700555 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1381/2295] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:00 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:01.717958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d3d55635-52f2-4470-8d4e-514ddd4597e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:01.718591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d3d55635-52f2-4470-8d4e-514ddd4597e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:01.724576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d3d55635-52f2-4470-8d4e-514ddd4597e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:01.725062 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d3d55635-52f2-4470-8d4e-514ddd4597e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:01.725801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d3d55635-52f2-4470-8d4e-514ddd4597e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:01.726231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d3d55635-52f2-4470-8d4e-514ddd4597e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:01.853753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d3d55635-52f2-4470-8d4e-514ddd4597e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:01.863181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d3d55635-52f2-4470-8d4e-514ddd4597e6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149178 Jun 10 13:32:01.863749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1382/2296] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:01 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:02.879558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-26414d7e-5787-48d8-9e7a-9e379a77b51b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:02.880221 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-26414d7e-5787-48d8-9e7a-9e379a77b51b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:02.886566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26414d7e-5787-48d8-9e7a-9e379a77b51b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:02.887020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26414d7e-5787-48d8-9e7a-9e379a77b51b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:02.887772 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26414d7e-5787-48d8-9e7a-9e379a77b51b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:02.888207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26414d7e-5787-48d8-9e7a-9e379a77b51b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:03.010998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-26414d7e-5787-48d8-9e7a-9e379a77b51b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:03.018771 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-26414d7e-5787-48d8-9e7a-9e379a77b51b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.143207 Jun 10 13:32:03.019334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1383/2297] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:02 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:04.035387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1263dfbc-0f6a-431c-bc9b-601ee9ed9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:04.036031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1263dfbc-0f6a-431c-bc9b-601ee9ed9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:04.042477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1263dfbc-0f6a-431c-bc9b-601ee9ed9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:04.042916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1263dfbc-0f6a-431c-bc9b-601ee9ed9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:04.043656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1263dfbc-0f6a-431c-bc9b-601ee9ed9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:04.044084 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1263dfbc-0f6a-431c-bc9b-601ee9ed9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:04.167768 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1263dfbc-0f6a-431c-bc9b-601ee9ed9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:04.175642 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1263dfbc-0f6a-431c-bc9b-601ee9ed9e17 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144088 Jun 10 13:32:04.176208 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1384/2298] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:04 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:05.192554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3cc3212e-c146-4d51-a0c9-1995e7aeb2dc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:05.193243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3cc3212e-c146-4d51-a0c9-1995e7aeb2dc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:05.199598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3cc3212e-c146-4d51-a0c9-1995e7aeb2dc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:05.200039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3cc3212e-c146-4d51-a0c9-1995e7aeb2dc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:05.200797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3cc3212e-c146-4d51-a0c9-1995e7aeb2dc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:05.201235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3cc3212e-c146-4d51-a0c9-1995e7aeb2dc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:05.328891 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3cc3212e-c146-4d51-a0c9-1995e7aeb2dc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:05.337152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3cc3212e-c146-4d51-a0c9-1995e7aeb2dc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148374 Jun 10 13:32:05.337740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1385/2299] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:05 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:06.354745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8a749855-8ff5-4eb3-97f0-d36d8cd7c303 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:06.355359 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8a749855-8ff5-4eb3-97f0-d36d8cd7c303 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:06.361810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a749855-8ff5-4eb3-97f0-d36d8cd7c303 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:06.362259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a749855-8ff5-4eb3-97f0-d36d8cd7c303 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:06.363027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a749855-8ff5-4eb3-97f0-d36d8cd7c303 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:06.363455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a749855-8ff5-4eb3-97f0-d36d8cd7c303 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:06.485871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8a749855-8ff5-4eb3-97f0-d36d8cd7c303 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:06.493813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8a749855-8ff5-4eb3-97f0-d36d8cd7c303 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.142952 Jun 10 13:32:06.494380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1386/2300] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:06 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:07.510708 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-73e2d0fa-7c97-4047-a4aa-1671a2da8755 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:07.511287 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-73e2d0fa-7c97-4047-a4aa-1671a2da8755 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:07.517820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-73e2d0fa-7c97-4047-a4aa-1671a2da8755 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:07.518264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-73e2d0fa-7c97-4047-a4aa-1671a2da8755 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:07.519024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-73e2d0fa-7c97-4047-a4aa-1671a2da8755 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:07.519455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-73e2d0fa-7c97-4047-a4aa-1671a2da8755 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:07.647262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-73e2d0fa-7c97-4047-a4aa-1671a2da8755 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:07.655827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-73e2d0fa-7c97-4047-a4aa-1671a2da8755 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149132 Jun 10 13:32:07.656391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1387/2301] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:07 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:08.673310 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ff14720c-0d01-468c-a86f-bbedfbc8f65d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:08.674387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ff14720c-0d01-468c-a86f-bbedfbc8f65d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:08.681434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff14720c-0d01-468c-a86f-bbedfbc8f65d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:08.681875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff14720c-0d01-468c-a86f-bbedfbc8f65d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:08.683502 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff14720c-0d01-468c-a86f-bbedfbc8f65d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:08.683934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ff14720c-0d01-468c-a86f-bbedfbc8f65d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:08.810938 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ff14720c-0d01-468c-a86f-bbedfbc8f65d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:08.818875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ff14720c-0d01-468c-a86f-bbedfbc8f65d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.149391 Jun 10 13:32:08.820945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1388/2302] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:08 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:09.836380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0c328f5a-dd49-4143-86ae-350999839cb0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:09.837374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0c328f5a-dd49-4143-86ae-350999839cb0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:09.844990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c328f5a-dd49-4143-86ae-350999839cb0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:09.845475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c328f5a-dd49-4143-86ae-350999839cb0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:09.846311 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c328f5a-dd49-4143-86ae-350999839cb0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:09.846744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c328f5a-dd49-4143-86ae-350999839cb0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:09.850982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:32:09.974116 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0c328f5a-dd49-4143-86ae-350999839cb0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:09.982800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0c328f5a-dd49-4143-86ae-350999839cb0 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150290 Jun 10 13:32:09.983370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1389/2303] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:09 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:10.999567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5af6cce3-2f5d-452f-9764-d5383cc5dbff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:11.000266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5af6cce3-2f5d-452f-9764-d5383cc5dbff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:11.006721 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5af6cce3-2f5d-452f-9764-d5383cc5dbff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:11.007170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5af6cce3-2f5d-452f-9764-d5383cc5dbff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:11.007925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5af6cce3-2f5d-452f-9764-d5383cc5dbff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:11.008353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5af6cce3-2f5d-452f-9764-d5383cc5dbff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:11.146769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5af6cce3-2f5d-452f-9764-d5383cc5dbff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:11.154650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5af6cce3-2f5d-452f-9764-d5383cc5dbff tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.159004 Jun 10 13:32:11.155212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1390/2304] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:10 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:12.172025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7a741381-4f11-41b2-bf2d-13fe9067460e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:12.172698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7a741381-4f11-41b2-bf2d-13fe9067460e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:12.179102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a741381-4f11-41b2-bf2d-13fe9067460e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:12.179551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a741381-4f11-41b2-bf2d-13fe9067460e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:12.180304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a741381-4f11-41b2-bf2d-13fe9067460e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:12.180794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7a741381-4f11-41b2-bf2d-13fe9067460e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:12.308746 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7a741381-4f11-41b2-bf2d-13fe9067460e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:12.316757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7a741381-4f11-41b2-bf2d-13fe9067460e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.148596 Jun 10 13:32:12.317435 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1391/2305] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:12 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:13.333957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-697202bb-526d-41e4-8625-4aa4ca30f1c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:13.334653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-697202bb-526d-41e4-8625-4aa4ca30f1c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:13.341196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-697202bb-526d-41e4-8625-4aa4ca30f1c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:13.341637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-697202bb-526d-41e4-8625-4aa4ca30f1c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:13.342385 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-697202bb-526d-41e4-8625-4aa4ca30f1c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:13.342811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-697202bb-526d-41e4-8625-4aa4ca30f1c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:13.468333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-697202bb-526d-41e4-8625-4aa4ca30f1c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:13.476080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-697202bb-526d-41e4-8625-4aa4ca30f1c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146191 Jun 10 13:32:13.476651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1392/2306] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:13 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:14.492894 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-130149b1-46a6-4be2-b48e-7b2145d367b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:14.493547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-130149b1-46a6-4be2-b48e-7b2145d367b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:14.500647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-130149b1-46a6-4be2-b48e-7b2145d367b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:14.501123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-130149b1-46a6-4be2-b48e-7b2145d367b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:14.501872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-130149b1-46a6-4be2-b48e-7b2145d367b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:14.502306 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-130149b1-46a6-4be2-b48e-7b2145d367b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:14.625711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-130149b1-46a6-4be2-b48e-7b2145d367b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:14.633605 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-130149b1-46a6-4be2-b48e-7b2145d367b6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.144610 Jun 10 13:32:14.634169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1393/2307] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:14 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:15.650974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ac178b03-238a-4d1c-992d-6fdc332e6622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:15.651618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ac178b03-238a-4d1c-992d-6fdc332e6622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:15.657948 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ac178b03-238a-4d1c-992d-6fdc332e6622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:15.658390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ac178b03-238a-4d1c-992d-6fdc332e6622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:15.659134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ac178b03-238a-4d1c-992d-6fdc332e6622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:15.659563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ac178b03-238a-4d1c-992d-6fdc332e6622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:15.789678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ac178b03-238a-4d1c-992d-6fdc332e6622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:15.798195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ac178b03-238a-4d1c-992d-6fdc332e6622 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.151624 Jun 10 13:32:15.798758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1394/2308] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:15 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:16.815179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-80812372-8e18-4565-8bf4-b7a26db06a35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:16.815812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-80812372-8e18-4565-8bf4-b7a26db06a35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:16.822343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80812372-8e18-4565-8bf4-b7a26db06a35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:16.822789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80812372-8e18-4565-8bf4-b7a26db06a35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:16.823541 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80812372-8e18-4565-8bf4-b7a26db06a35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:16.823971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-80812372-8e18-4565-8bf4-b7a26db06a35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:16.949487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-80812372-8e18-4565-8bf4-b7a26db06a35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:16.957599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-80812372-8e18-4565-8bf4-b7a26db06a35 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.146277 Jun 10 13:32:16.959202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1395/2309] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:16 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:17.975157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-55ce4e8e-feaf-4f1d-8f23-fc2111766e56 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:17.975795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-55ce4e8e-feaf-4f1d-8f23-fc2111766e56 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:17.982207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55ce4e8e-feaf-4f1d-8f23-fc2111766e56 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:17.982656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55ce4e8e-feaf-4f1d-8f23-fc2111766e56 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:17.983883 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55ce4e8e-feaf-4f1d-8f23-fc2111766e56 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:17.984551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-55ce4e8e-feaf-4f1d-8f23-fc2111766e56 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:18.113643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-55ce4e8e-feaf-4f1d-8f23-fc2111766e56 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:18.122199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-55ce4e8e-feaf-4f1d-8f23-fc2111766e56 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.150864 Jun 10 13:32:18.122763 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1396/2310] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:17 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:32:18.127166 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ea525568-a962-4e43-89b8-2a99f0d02c76 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:18.127655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ea525568-a962-4e43-89b8-2a99f0d02c76 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:18.133915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ea525568-a962-4e43-89b8-2a99f0d02c76 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:18.134366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ea525568-a962-4e43-89b8-2a99f0d02c76 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:18.135154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ea525568-a962-4e43-89b8-2a99f0d02c76 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:18.135593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ea525568-a962-4e43-89b8-2a99f0d02c76 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:18.150478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-ea525568-a962-4e43-89b8-2a99f0d02c76 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] HTTP exception thrown: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ea525568-a962-4e43-89b8-2a99f0d02c76 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Request method failure captured: Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a HTTP/1.1 Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0GLIJPQ8lSWSCxXU9neIcS9fWNdW4XJF-omQVn8xvgfajtrPtR9xT7yJRQ7uDkAeVSjjX8Zr-TojYB67_YvghXuxNAaWo9VFXLD41gtOICuurNBjdxNv9Y58tQmvsCSdX2FomGdBsssQdaFXDYnlp0d7vxIpJLVub7cC52vynSBvwE Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 6dace38cecd44f87becbf6dc213851b3 Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:32:18.152181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:32:18.153931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_6dace38cecd44f87becbf6dc213851b3", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:32:18.153931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:32:18.153931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 6dace38cecd44f87becbf6dc213851b3 Jun 10 13:32:18.153931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:32:18.153931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeTestJSON-1599090335-project Jun 10 13:32:18.153931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:32:18.153931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:32:18.153931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 9c8b2c27b69a4776ab6b7b7820da02d9 Jun 10 13:32:18.153931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeTestJSON-1599090335-project Jun 10 13:32:18.153931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:32:18.153931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:32:18.153931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '9e403cf8-5aee-49f3-bbbc-600263bd148a'} Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:32:18.158002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:32:18.166919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:32:18.168591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ea525568-a962-4e43-89b8-2a99f0d02c76 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Returning 404 to user: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:32:18.168591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ea525568-a962-4e43-89b8-2a99f0d02c76 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "DELETE /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 404 len: 111 microversion: 2.1 time: 0.029943 Jun 10 13:32:18.168591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1397/2311] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:32:18 2022] DELETE /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:32:18.168591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-99d371d3-54eb-4c77-919b-ad529b86b042 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:18.168591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-99d371d3-54eb-4c77-919b-ad529b86b042 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] [instance: 9e403cf8-5aee-49f3-bbbc-600263bd148a] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:18.171677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99d371d3-54eb-4c77-919b-ad529b86b042 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:18.172130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99d371d3-54eb-4c77-919b-ad529b86b042 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:18.172992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99d371d3-54eb-4c77-919b-ad529b86b042 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:18.173487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99d371d3-54eb-4c77-919b-ad529b86b042 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:18.188152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-99d371d3-54eb-4c77-919b-ad529b86b042 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] HTTP exception thrown: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-99d371d3-54eb-4c77-919b-ad529b86b042 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Request method failure captured: Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a HTTP/1.1 Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0GLIJPQ8lSWSCxXU9neIcS9fWNdW4XJF-omQVn8xvgfajtrPtR9xT7yJRQ7uDkAeVSjjX8Zr-TojYB67_YvghXuxNAaWo9VFXLD41gtOICuurNBjdxNv9Y58tQmvsCSdX2FomGdBsssQdaFXDYnlp0d7vxIpJLVub7cC52vynSBvwE Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 6dace38cecd44f87becbf6dc213851b3 Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:32:18.189733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:32:18.191358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_6dace38cecd44f87becbf6dc213851b3", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6dace38cecd44f87becbf6dc213851b3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:32:18.191358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:32:18.191358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 6dace38cecd44f87becbf6dc213851b3 Jun 10 13:32:18.191358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1599090335 Jun 10 13:32:18.191358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeTestJSON-1599090335-project Jun 10 13:32:18.191358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:32:18.191358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:32:18.191358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 9c8b2c27b69a4776ab6b7b7820da02d9 Jun 10 13:32:18.191358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeTestJSON-1599090335-project Jun 10 13:32:18.191358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:32:18.191358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:32:18.191358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '9e403cf8-5aee-49f3-bbbc-600263bd148a'} Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:32:18.192334 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. Jun 10 13:32:18.193628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:32:18.194769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-99d371d3-54eb-4c77-919b-ad529b86b042 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] Returning 404 to user: Instance 9e403cf8-5aee-49f3-bbbc-600263bd148a could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:32:18.194769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-99d371d3-54eb-4c77-919b-ad529b86b042 tempest-AttachVolumeTestJSON-1599090335 tempest-AttachVolumeTestJSON-1599090335-project] 10.222.0.25 "GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a" status: 404 len: 111 microversion: 2.1 time: 0.032670 Jun 10 13:32:18.194769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1398/2312] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:18 2022] GET /compute/v2.1/servers/9e403cf8-5aee-49f3-bbbc-600263bd148a => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:32:19.194628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d503a07c-7da9-4188-8fde-ca0619bc80ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:19.196073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d503a07c-7da9-4188-8fde-ca0619bc80ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:19.206087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d503a07c-7da9-4188-8fde-ca0619bc80ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:19.206836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d503a07c-7da9-4188-8fde-ca0619bc80ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:19.207830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d503a07c-7da9-4188-8fde-ca0619bc80ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:19.208512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d503a07c-7da9-4188-8fde-ca0619bc80ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:19.501532 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d503a07c-7da9-4188-8fde-ca0619bc80ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:19.512041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d503a07c-7da9-4188-8fde-ca0619bc80ad tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1559 microversion: 2.1 time: 0.323686 Jun 10 13:32:19.518184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1399/2313] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:19 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1559 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:19.528614 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-681f1935-8146-46c1-9b91-29e8070539da req-fc9c67b5-2272-4916-93f1-d5a15ca94cf8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744", "name": "network-vif-deleted", "tag": "46531dd1-3591-49bf-b170-0a5b54b53fcd"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:32:19.537195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-681f1935-8146-46c1-9b91-29e8070539da req-fc9c67b5-2272-4916-93f1-d5a15ca94cf8 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:19.537637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-681f1935-8146-46c1-9b91-29e8070539da req-fc9c67b5-2272-4916-93f1-d5a15ca94cf8 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:19.549293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-681f1935-8146-46c1-9b91-29e8070539da req-fc9c67b5-2272-4916-93f1-d5a15ca94cf8 service nova] Creating event network-vif-deleted:46531dd1-3591-49bf-b170-0a5b54b53fcd for instance a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 on hv-cinder-maste Jun 10 13:32:19.554335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-681f1935-8146-46c1-9b91-29e8070539da req-fc9c67b5-2272-4916-93f1-d5a15ca94cf8 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030140 Jun 10 13:32:19.554908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1400/2314] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:32:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:32:20.531609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b187292c-17ec-4e4b-bd6b-d3959162a9c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:20.532435 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b187292c-17ec-4e4b-bd6b-d3959162a9c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:20.540324 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b187292c-17ec-4e4b-bd6b-d3959162a9c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:20.541650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b187292c-17ec-4e4b-bd6b-d3959162a9c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:20.543061 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b187292c-17ec-4e4b-bd6b-d3959162a9c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:20.543172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b187292c-17ec-4e4b-bd6b-d3959162a9c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:20.606944 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b187292c-17ec-4e4b-bd6b-d3959162a9c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:20.615072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b187292c-17ec-4e4b-bd6b-d3959162a9c6 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.088279 Jun 10 13:32:20.615647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1401/2315] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:20 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:21.693763 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a3114672-2fdb-4c7b-9e34-7aabffbb1162 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:21.694464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a3114672-2fdb-4c7b-9e34-7aabffbb1162 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:21.702795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3114672-2fdb-4c7b-9e34-7aabffbb1162 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:21.703094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3114672-2fdb-4c7b-9e34-7aabffbb1162 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:21.704359 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3114672-2fdb-4c7b-9e34-7aabffbb1162 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:21.704535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a3114672-2fdb-4c7b-9e34-7aabffbb1162 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:21.771708 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a3114672-2fdb-4c7b-9e34-7aabffbb1162 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:21.780137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a3114672-2fdb-4c7b-9e34-7aabffbb1162 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.151348 Jun 10 13:32:21.780826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1402/2316] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:21 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:22.797594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dd80d820-7b36-4d4f-b1db-1c5b949137de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:22.798231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dd80d820-7b36-4d4f-b1db-1c5b949137de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:22.804609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd80d820-7b36-4d4f-b1db-1c5b949137de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:22.805084 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd80d820-7b36-4d4f-b1db-1c5b949137de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:22.805857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd80d820-7b36-4d4f-b1db-1c5b949137de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:22.806923 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd80d820-7b36-4d4f-b1db-1c5b949137de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:22.866902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dd80d820-7b36-4d4f-b1db-1c5b949137de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:22.875258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dd80d820-7b36-4d4f-b1db-1c5b949137de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.081529 Jun 10 13:32:22.875811 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1403/2317] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:22 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:23.891601 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-eb727eee-1d84-4a26-9763-c159ef2fd8b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:23.892238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-eb727eee-1d84-4a26-9763-c159ef2fd8b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:23.898722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eb727eee-1d84-4a26-9763-c159ef2fd8b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:23.899173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eb727eee-1d84-4a26-9763-c159ef2fd8b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:23.899941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eb727eee-1d84-4a26-9763-c159ef2fd8b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:23.900380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-eb727eee-1d84-4a26-9763-c159ef2fd8b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:23.971996 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-eb727eee-1d84-4a26-9763-c159ef2fd8b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:23.979938 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-eb727eee-1d84-4a26-9763-c159ef2fd8b8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.092181 Jun 10 13:32:23.980834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1404/2318] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:23 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:24.997841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f8f694f8-4441-49e0-bcb3-5cae3efe7a8d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:24.998461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f8f694f8-4441-49e0-bcb3-5cae3efe7a8d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:25.006198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8f694f8-4441-49e0-bcb3-5cae3efe7a8d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:25.006621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8f694f8-4441-49e0-bcb3-5cae3efe7a8d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:25.007450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8f694f8-4441-49e0-bcb3-5cae3efe7a8d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:25.007879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f8f694f8-4441-49e0-bcb3-5cae3efe7a8d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:25.073192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f8f694f8-4441-49e0-bcb3-5cae3efe7a8d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:25.081536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f8f694f8-4441-49e0-bcb3-5cae3efe7a8d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.087606 Jun 10 13:32:25.082301 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1405/2319] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:24 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:26.099079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-35e7a284-41db-45ff-975f-cb0bc3c24cf3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:26.099749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-35e7a284-41db-45ff-975f-cb0bc3c24cf3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:26.106977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35e7a284-41db-45ff-975f-cb0bc3c24cf3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:26.107661 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35e7a284-41db-45ff-975f-cb0bc3c24cf3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:26.108436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35e7a284-41db-45ff-975f-cb0bc3c24cf3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:26.108957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35e7a284-41db-45ff-975f-cb0bc3c24cf3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:26.179867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-35e7a284-41db-45ff-975f-cb0bc3c24cf3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:26.187763 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-35e7a284-41db-45ff-975f-cb0bc3c24cf3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.092673 Jun 10 13:32:26.188493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1406/2320] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:26 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:27.204243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-00e72254-24d9-4c43-b2d7-ddb9fcffb875 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:27.206167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-00e72254-24d9-4c43-b2d7-ddb9fcffb875 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:27.214204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00e72254-24d9-4c43-b2d7-ddb9fcffb875 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:27.214805 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00e72254-24d9-4c43-b2d7-ddb9fcffb875 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:27.215683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00e72254-24d9-4c43-b2d7-ddb9fcffb875 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:27.216269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00e72254-24d9-4c43-b2d7-ddb9fcffb875 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:27.284461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-00e72254-24d9-4c43-b2d7-ddb9fcffb875 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:27.289144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:32:27.294821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-00e72254-24d9-4c43-b2d7-ddb9fcffb875 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.094813 Jun 10 13:32:27.297647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1407/2321] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:27 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:28.312734 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1a525d0b-1d4b-451a-b02c-a701ecdcbcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:28.313386 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1a525d0b-1d4b-451a-b02c-a701ecdcbcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:28.317185 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=58 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:32:28.322068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a525d0b-1d4b-451a-b02c-a701ecdcbcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:28.322358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a525d0b-1d4b-451a-b02c-a701ecdcbcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:28.322880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a525d0b-1d4b-451a-b02c-a701ecdcbcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:28.323504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a525d0b-1d4b-451a-b02c-a701ecdcbcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:28.407192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1a525d0b-1d4b-451a-b02c-a701ecdcbcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:28.416025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1a525d0b-1d4b-451a-b02c-a701ecdcbcca tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.107702 Jun 10 13:32:28.416601 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1408/2322] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:28 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:29.265022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2a62d956-73b9-42e6-a08a-1dc0fcac8ce3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Action: 'create', calling method: .version_select at 0x7f930524a3a0>, body: {"keypair": {"name": "tempest-keypair-1706060724", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMgYXX0QmWJ1ozBZcuRtCwSx2//heMEfM4XUrxJNoEFORBh3arfjfhNoBwdEed1bMo34FlymEl5eHi+z0J4gIQ/gZZ12ATqeehBtN8k7PFltADdkBJQlpmIoYsoqcdGBSw=="}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:32:29.279992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-2a62d956-73b9-42e6-a08a-1dc0fcac8ce3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Getting quotas for project eee8e4f43f484abdb588a8f3966d74ad. Resources: {'key_pairs'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:32:29.284482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-2a62d956-73b9-42e6-a08a-1dc0fcac8ce3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Getting quotas for user 434ae422b68e4145ad8ecac27f450cbc and project eee8e4f43f484abdb588a8f3966d74ad. Resources: {'key_pairs'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:32:29.363271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2a62d956-73b9-42e6-a08a-1dc0fcac8ce3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.243800 Jun 10 13:32:29.363939 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1409/2323] 10.222.0.25 () {60 vars in 1158 bytes} [Fri Jun 10 13:32:29 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:32:29.434361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-adc87caa-2998-4722-88f0-0167138fc8d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:29.435034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-adc87caa-2998-4722-88f0-0167138fc8d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:29.441262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-adc87caa-2998-4722-88f0-0167138fc8d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:29.441710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-adc87caa-2998-4722-88f0-0167138fc8d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:29.442464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-adc87caa-2998-4722-88f0-0167138fc8d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:29.442892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-adc87caa-2998-4722-88f0-0167138fc8d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:29.511242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-adc87caa-2998-4722-88f0-0167138fc8d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:29.519565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-adc87caa-2998-4722-88f0-0167138fc8d7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.089710 Jun 10 13:32:29.528863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1410/2324] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:29 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:32:30.536784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1ca8928c-5582-4dec-8f04-1f1a13729e18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:30.537419 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1ca8928c-5582-4dec-8f04-1f1a13729e18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:30.543942 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ca8928c-5582-4dec-8f04-1f1a13729e18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:30.544396 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ca8928c-5582-4dec-8f04-1f1a13729e18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:30.545187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ca8928c-5582-4dec-8f04-1f1a13729e18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:30.545617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1ca8928c-5582-4dec-8f04-1f1a13729e18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:30.612898 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1ca8928c-5582-4dec-8f04-1f1a13729e18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:30.621391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1ca8928c-5582-4dec-8f04-1f1a13729e18 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.088521 Jun 10 13:32:30.622214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1411/2325] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:30 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:31.593678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1899657197", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e47293bb-c3a4-4f5c-acfc-80db56fb3d26"}], "security_groups": [{"name": "tempest-securitygroup--1365078546"}], "key_name": "tempest-keypair-1706060724", "user_data": "}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:32:31.698913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.network.neutron [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] validate_networks() for [('e47293bb-c3a4-4f5c-acfc-80db56fb3d26', None, None, None, None, None)] {{(pid=118092) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:32:32.078367 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Flavor limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:32:32.078794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Image limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:32:32.079530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Flavor pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:32:32.079936 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Image pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:32:32.080367 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:32:32.089977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:32.090426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:32.091220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:32.091654 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:32.111162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Getting quotas for project eee8e4f43f484abdb588a8f3966d74ad. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:32:32.115294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Getting quotas for user 434ae422b68e4145ad8ecac27f450cbc and project eee8e4f43f484abdb588a8f3966d74ad. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:32:32.127438 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Going to run 1 instances... {{(pid=118092) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:32:32.129675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118092) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:32:32.261532 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-40c4b13b-d419-496d-83c3-6c352a6dcf07 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.672069 Jun 10 13:32:32.262184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1412/2326] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:32:31 2022] POST /compute/v2.1/servers => generated 402 bytes in 673 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jun 10 13:32:32.272057 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e68f0b84-5f47-4252-8ed4-2fe3f38dfed8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:32.272930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e68f0b84-5f47-4252-8ed4-2fe3f38dfed8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:32.279152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e68f0b84-5f47-4252-8ed4-2fe3f38dfed8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:32.279609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e68f0b84-5f47-4252-8ed4-2fe3f38dfed8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:32.280390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e68f0b84-5f47-4252-8ed4-2fe3f38dfed8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:32.280910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e68f0b84-5f47-4252-8ed4-2fe3f38dfed8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:32.349276 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e68f0b84-5f47-4252-8ed4-2fe3f38dfed8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:32.357976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e68f0b84-5f47-4252-8ed4-2fe3f38dfed8 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.089624 Jun 10 13:32:32.358558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1413/2327] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:32 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:32:32.364588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-35e13a07-4f8e-4547-a8ef-a2b14996dc7f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:32.365285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-35e13a07-4f8e-4547-a8ef-a2b14996dc7f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:32.419890 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-35e13a07-4f8e-4547-a8ef-a2b14996dc7f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:32.428313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-35e13a07-4f8e-4547-a8ef-a2b14996dc7f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1242 microversion: 2.1 time: 0.067542 Jun 10 13:32:32.429183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1414/2328] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:32 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1242 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:32:33.374957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dbc15ae2-a688-4a41-9541-b18ebf01f7ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:33.375587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dbc15ae2-a688-4a41-9541-b18ebf01f7ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:33.383019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dbc15ae2-a688-4a41-9541-b18ebf01f7ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:33.383467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dbc15ae2-a688-4a41-9541-b18ebf01f7ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:33.388541 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dbc15ae2-a688-4a41-9541-b18ebf01f7ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:33.388541 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dbc15ae2-a688-4a41-9541-b18ebf01f7ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:33.450990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dbc15ae2-a688-4a41-9541-b18ebf01f7ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:33.459330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dbc15ae2-a688-4a41-9541-b18ebf01f7ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.088484 Jun 10 13:32:33.463464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1415/2329] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:33 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:33.464843 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ebb60a3a-3d35-42e7-a6a3-8dd9e3db6045 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:33.465438 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ebb60a3a-3d35-42e7-a6a3-8dd9e3db6045 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:33.471030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebb60a3a-3d35-42e7-a6a3-8dd9e3db6045 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:33.471472 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebb60a3a-3d35-42e7-a6a3-8dd9e3db6045 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:33.472353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebb60a3a-3d35-42e7-a6a3-8dd9e3db6045 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:33.472867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ebb60a3a-3d35-42e7-a6a3-8dd9e3db6045 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:33.531658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ebb60a3a-3d35-42e7-a6a3-8dd9e3db6045 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:33.539217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ebb60a3a-3d35-42e7-a6a3-8dd9e3db6045 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1246 microversion: 2.1 time: 0.078396 Jun 10 13:32:33.539771 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1416/2330] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:33 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1246 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:32:34.478360 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a7dcb7e5-d1b1-4106-9eb4-3292b25307de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:34.478996 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a7dcb7e5-d1b1-4106-9eb4-3292b25307de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:34.486151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7dcb7e5-d1b1-4106-9eb4-3292b25307de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:34.486590 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7dcb7e5-d1b1-4106-9eb4-3292b25307de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:34.487341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7dcb7e5-d1b1-4106-9eb4-3292b25307de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:34.487770 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a7dcb7e5-d1b1-4106-9eb4-3292b25307de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:34.552962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a7dcb7e5-d1b1-4106-9eb4-3292b25307de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:34.562188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a7dcb7e5-d1b1-4106-9eb4-3292b25307de tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.088827 Jun 10 13:32:34.562784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1417/2331] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:34 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:34.567515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ccfbf5b6-bc83-4903-9f72-5ddd799b0b50 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:34.568109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ccfbf5b6-bc83-4903-9f72-5ddd799b0b50 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:34.574535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ccfbf5b6-bc83-4903-9f72-5ddd799b0b50 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:34.574986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ccfbf5b6-bc83-4903-9f72-5ddd799b0b50 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:34.575759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ccfbf5b6-bc83-4903-9f72-5ddd799b0b50 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:34.576185 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ccfbf5b6-bc83-4903-9f72-5ddd799b0b50 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:34.638972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ccfbf5b6-bc83-4903-9f72-5ddd799b0b50 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:34.646749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ccfbf5b6-bc83-4903-9f72-5ddd799b0b50 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1302 microversion: 2.1 time: 0.082929 Jun 10 13:32:34.650333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1418/2332] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:34 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1302 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:32:35.578745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-959690a3-6794-4347-b2ea-47b782cf8b4d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:35.579650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-959690a3-6794-4347-b2ea-47b782cf8b4d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:35.586112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-959690a3-6794-4347-b2ea-47b782cf8b4d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:35.586559 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-959690a3-6794-4347-b2ea-47b782cf8b4d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:35.587434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-959690a3-6794-4347-b2ea-47b782cf8b4d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:35.587875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-959690a3-6794-4347-b2ea-47b782cf8b4d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:35.651239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-959690a3-6794-4347-b2ea-47b782cf8b4d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:35.659318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-959690a3-6794-4347-b2ea-47b782cf8b4d tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.084620 Jun 10 13:32:35.659888 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1419/2333] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:35 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:35.666954 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-646cd3e9-b69d-4f0e-87e8-d4a1fb8885c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:35.667668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-646cd3e9-b69d-4f0e-87e8-d4a1fb8885c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:35.673841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-646cd3e9-b69d-4f0e-87e8-d4a1fb8885c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:35.674229 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-646cd3e9-b69d-4f0e-87e8-d4a1fb8885c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:35.674981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-646cd3e9-b69d-4f0e-87e8-d4a1fb8885c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:35.675412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-646cd3e9-b69d-4f0e-87e8-d4a1fb8885c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:36.011702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-646cd3e9-b69d-4f0e-87e8-d4a1fb8885c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:36.019381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-646cd3e9-b69d-4f0e-87e8-d4a1fb8885c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1368 microversion: 2.1 time: 0.358474 Jun 10 13:32:36.019956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1420/2334] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:35 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1368 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:32:36.258718 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-1ec914b9-bf76-47b9-8e57-67fa60c42bb4 req-f1167e38-e0a3-4426-a48e-0a8af8a7970e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b923ce4b-4493-492c-82b8-4adc45ea47f7", "tag": "f237b5cf-c7a4-49a8-b20d-f2f6c15cb9af"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:32:36.272578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-1ec914b9-bf76-47b9-8e57-67fa60c42bb4 req-f1167e38-e0a3-4426-a48e-0a8af8a7970e service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:36.273360 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-1ec914b9-bf76-47b9-8e57-67fa60c42bb4 req-f1167e38-e0a3-4426-a48e-0a8af8a7970e service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:36.286681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-1ec914b9-bf76-47b9-8e57-67fa60c42bb4 req-f1167e38-e0a3-4426-a48e-0a8af8a7970e service nova] Creating event network-changed:f237b5cf-c7a4-49a8-b20d-f2f6c15cb9af for instance b923ce4b-4493-492c-82b8-4adc45ea47f7 on hv-cinder-maste Jun 10 13:32:36.291821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-1ec914b9-bf76-47b9-8e57-67fa60c42bb4 req-f1167e38-e0a3-4426-a48e-0a8af8a7970e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037311 Jun 10 13:32:36.292602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1421/2335] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:32:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:32:36.676581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ded000fe-ecdd-40b8-8e67-9df51bcc86a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:36.677328 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ded000fe-ecdd-40b8-8e67-9df51bcc86a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:36.684359 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ded000fe-ecdd-40b8-8e67-9df51bcc86a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:36.684824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ded000fe-ecdd-40b8-8e67-9df51bcc86a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:36.685577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ded000fe-ecdd-40b8-8e67-9df51bcc86a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:36.686006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ded000fe-ecdd-40b8-8e67-9df51bcc86a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:36.749917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ded000fe-ecdd-40b8-8e67-9df51bcc86a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:36.757884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ded000fe-ecdd-40b8-8e67-9df51bcc86a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.085995 Jun 10 13:32:36.758636 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1422/2336] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:36 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:37.037235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-69a15152-f97a-4a6f-ab97-ebad13606cb8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:37.037843 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-69a15152-f97a-4a6f-ab97-ebad13606cb8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:37.044327 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-69a15152-f97a-4a6f-ab97-ebad13606cb8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:37.044795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-69a15152-f97a-4a6f-ab97-ebad13606cb8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:37.045669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-69a15152-f97a-4a6f-ab97-ebad13606cb8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:37.046102 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-69a15152-f97a-4a6f-ab97-ebad13606cb8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:37.173037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-69a15152-f97a-4a6f-ab97-ebad13606cb8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:37.180477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-69a15152-f97a-4a6f-ab97-ebad13606cb8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1368 microversion: 2.1 time: 0.147179 Jun 10 13:32:37.181195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1423/2337] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:37 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1368 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:37.775515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c7af1ed5-8d62-4018-a228-6983d886830e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:37.776121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c7af1ed5-8d62-4018-a228-6983d886830e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:37.782597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7af1ed5-8d62-4018-a228-6983d886830e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:37.783042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7af1ed5-8d62-4018-a228-6983d886830e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:37.783800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7af1ed5-8d62-4018-a228-6983d886830e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:37.784232 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c7af1ed5-8d62-4018-a228-6983d886830e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:37.843790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c7af1ed5-8d62-4018-a228-6983d886830e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:37.851690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c7af1ed5-8d62-4018-a228-6983d886830e tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.080029 Jun 10 13:32:37.852250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1424/2338] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:37 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:38.197860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-24eca36a-2a91-4400-8540-912c31fc34d9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:38.198504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-24eca36a-2a91-4400-8540-912c31fc34d9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:38.204911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-24eca36a-2a91-4400-8540-912c31fc34d9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:38.205362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-24eca36a-2a91-4400-8540-912c31fc34d9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:38.206111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-24eca36a-2a91-4400-8540-912c31fc34d9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:38.206544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-24eca36a-2a91-4400-8540-912c31fc34d9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:38.330591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-24eca36a-2a91-4400-8540-912c31fc34d9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:38.338791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-24eca36a-2a91-4400-8540-912c31fc34d9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1368 microversion: 2.1 time: 0.144763 Jun 10 13:32:38.339381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1425/2339] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:38 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1368 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:38.869224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-91ee62d3-ef57-422a-90de-31dcb748792c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:38.869837 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-91ee62d3-ef57-422a-90de-31dcb748792c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:38.876169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91ee62d3-ef57-422a-90de-31dcb748792c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:38.876616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91ee62d3-ef57-422a-90de-31dcb748792c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:38.877408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91ee62d3-ef57-422a-90de-31dcb748792c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:38.877845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91ee62d3-ef57-422a-90de-31dcb748792c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:38.941413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-91ee62d3-ef57-422a-90de-31dcb748792c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:38.949987 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-91ee62d3-ef57-422a-90de-31dcb748792c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.085228 Jun 10 13:32:38.950551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1426/2340] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:38 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:39.355625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-35b597d5-daf4-4b0a-a41e-94455215c505 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:39.356262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-35b597d5-daf4-4b0a-a41e-94455215c505 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:39.362594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35b597d5-daf4-4b0a-a41e-94455215c505 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:39.363044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35b597d5-daf4-4b0a-a41e-94455215c505 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:39.363818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35b597d5-daf4-4b0a-a41e-94455215c505 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:39.364247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35b597d5-daf4-4b0a-a41e-94455215c505 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:39.489109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-35b597d5-daf4-4b0a-a41e-94455215c505 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:39.492626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:32:39.497594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-35b597d5-daf4-4b0a-a41e-94455215c505 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1368 microversion: 2.1 time: 0.145878 Jun 10 13:32:39.498154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1427/2341] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:39 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1368 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:39.967911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-342ed96c-7837-499f-9c88-5ded4d712026 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:39.968528 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-342ed96c-7837-499f-9c88-5ded4d712026 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:39.974977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-342ed96c-7837-499f-9c88-5ded4d712026 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:39.975415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-342ed96c-7837-499f-9c88-5ded4d712026 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:39.976230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-342ed96c-7837-499f-9c88-5ded4d712026 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:39.976657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-342ed96c-7837-499f-9c88-5ded4d712026 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:40.036564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-342ed96c-7837-499f-9c88-5ded4d712026 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:40.044792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-342ed96c-7837-499f-9c88-5ded4d712026 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.081138 Jun 10 13:32:40.045372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1428/2342] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:39 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:40.515259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f22e2f37-e02e-4681-b82b-e82cb45b31fe tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:40.515866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f22e2f37-e02e-4681-b82b-e82cb45b31fe tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:40.521988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f22e2f37-e02e-4681-b82b-e82cb45b31fe tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:40.522437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f22e2f37-e02e-4681-b82b-e82cb45b31fe tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:40.523194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f22e2f37-e02e-4681-b82b-e82cb45b31fe tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:40.523628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f22e2f37-e02e-4681-b82b-e82cb45b31fe tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:40.668524 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f22e2f37-e02e-4681-b82b-e82cb45b31fe tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:40.676288 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f22e2f37-e02e-4681-b82b-e82cb45b31fe tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1368 microversion: 2.1 time: 0.164824 Jun 10 13:32:40.676911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1429/2343] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:40 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1368 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:41.061599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-68aa2d60-d000-4821-af9f-aa09987ca7c2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:41.062247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-68aa2d60-d000-4821-af9f-aa09987ca7c2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:41.068808 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68aa2d60-d000-4821-af9f-aa09987ca7c2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:41.069263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68aa2d60-d000-4821-af9f-aa09987ca7c2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:41.070028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68aa2d60-d000-4821-af9f-aa09987ca7c2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:41.070488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68aa2d60-d000-4821-af9f-aa09987ca7c2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:41.131006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-68aa2d60-d000-4821-af9f-aa09987ca7c2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:41.138899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-68aa2d60-d000-4821-af9f-aa09987ca7c2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.081150 Jun 10 13:32:41.139457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1430/2344] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:41 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:41.693869 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-62dfcd18-9333-41c1-80c8-2905447df38e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:41.694477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-62dfcd18-9333-41c1-80c8-2905447df38e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:41.700540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62dfcd18-9333-41c1-80c8-2905447df38e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:41.701017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62dfcd18-9333-41c1-80c8-2905447df38e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:41.701771 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62dfcd18-9333-41c1-80c8-2905447df38e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:41.702201 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62dfcd18-9333-41c1-80c8-2905447df38e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:41.841381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-62dfcd18-9333-41c1-80c8-2905447df38e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:41.848969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-62dfcd18-9333-41c1-80c8-2905447df38e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1368 microversion: 2.1 time: 0.158898 Jun 10 13:32:41.849547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1431/2345] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:41 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1368 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:42.156012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ca740d08-98dc-4db0-b92e-0f224883cea5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:42.156631 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ca740d08-98dc-4db0-b92e-0f224883cea5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:42.158966 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:32:42.163813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca740d08-98dc-4db0-b92e-0f224883cea5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:42.168049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca740d08-98dc-4db0-b92e-0f224883cea5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:42.168049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca740d08-98dc-4db0-b92e-0f224883cea5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:42.168049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ca740d08-98dc-4db0-b92e-0f224883cea5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:42.228737 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ca740d08-98dc-4db0-b92e-0f224883cea5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:42.236783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ca740d08-98dc-4db0-b92e-0f224883cea5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.084587 Jun 10 13:32:42.237357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1432/2346] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:42 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:42.618188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-53ec2a4e-65a8-4fb5-8d35-f0dae2c3042f req-db59b31b-d638-4ddc-b2cc-ddcdbac659e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b923ce4b-4493-492c-82b8-4adc45ea47f7", "name": "network-vif-plugged", "status": "completed", "tag": "f237b5cf-c7a4-49a8-b20d-f2f6c15cb9af"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:32:42.622360 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=12 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:32:42.627782 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-53ec2a4e-65a8-4fb5-8d35-f0dae2c3042f req-db59b31b-d638-4ddc-b2cc-ddcdbac659e4 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:42.628225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-53ec2a4e-65a8-4fb5-8d35-f0dae2c3042f req-db59b31b-d638-4ddc-b2cc-ddcdbac659e4 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:42.639442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-53ec2a4e-65a8-4fb5-8d35-f0dae2c3042f req-db59b31b-d638-4ddc-b2cc-ddcdbac659e4 service nova] Creating event network-vif-plugged:f237b5cf-c7a4-49a8-b20d-f2f6c15cb9af for instance b923ce4b-4493-492c-82b8-4adc45ea47f7 on hv-cinder-maste Jun 10 13:32:42.644563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-53ec2a4e-65a8-4fb5-8d35-f0dae2c3042f req-db59b31b-d638-4ddc-b2cc-ddcdbac659e4 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029957 Jun 10 13:32:42.645179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1433/2347] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:32:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:32:42.867561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-df1fed4b-6943-4b1e-8d00-7b012e63cde7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:42.868294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-df1fed4b-6943-4b1e-8d00-7b012e63cde7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:42.875184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df1fed4b-6943-4b1e-8d00-7b012e63cde7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:42.875637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df1fed4b-6943-4b1e-8d00-7b012e63cde7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:42.876432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df1fed4b-6943-4b1e-8d00-7b012e63cde7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:42.876900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-df1fed4b-6943-4b1e-8d00-7b012e63cde7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:43.008273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-df1fed4b-6943-4b1e-8d00-7b012e63cde7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:43.015824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-df1fed4b-6943-4b1e-8d00-7b012e63cde7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1368 microversion: 2.1 time: 0.152954 Jun 10 13:32:43.016399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1434/2348] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:42 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1368 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:43.253714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ba1ddc39-4778-4f4d-9976-80ae185a0631 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:43.254332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ba1ddc39-4778-4f4d-9976-80ae185a0631 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:43.261590 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba1ddc39-4778-4f4d-9976-80ae185a0631 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:43.262041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba1ddc39-4778-4f4d-9976-80ae185a0631 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:43.262792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba1ddc39-4778-4f4d-9976-80ae185a0631 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:43.263222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba1ddc39-4778-4f4d-9976-80ae185a0631 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:43.323982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ba1ddc39-4778-4f4d-9976-80ae185a0631 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:43.332500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ba1ddc39-4778-4f4d-9976-80ae185a0631 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.082725 Jun 10 13:32:43.333113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1435/2349] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:43 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:44.031819 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0ff1c566-ece6-4236-8952-569677bbf907 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:44.032345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0ff1c566-ece6-4236-8952-569677bbf907 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:44.038394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ff1c566-ece6-4236-8952-569677bbf907 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:44.038842 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ff1c566-ece6-4236-8952-569677bbf907 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:44.039593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ff1c566-ece6-4236-8952-569677bbf907 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:44.040027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ff1c566-ece6-4236-8952-569677bbf907 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:44.168382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0ff1c566-ece6-4236-8952-569677bbf907 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:44.175735 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0ff1c566-ece6-4236-8952-569677bbf907 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1368 microversion: 2.1 time: 0.147852 Jun 10 13:32:44.176302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1436/2350] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:44 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1368 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:44.349308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-228e0bf5-ae7f-4af8-83ec-11e97af6772a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:44.349917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-228e0bf5-ae7f-4af8-83ec-11e97af6772a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:44.356754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-228e0bf5-ae7f-4af8-83ec-11e97af6772a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:44.357228 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-228e0bf5-ae7f-4af8-83ec-11e97af6772a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:44.357979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-228e0bf5-ae7f-4af8-83ec-11e97af6772a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:44.358411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-228e0bf5-ae7f-4af8-83ec-11e97af6772a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:44.418681 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-228e0bf5-ae7f-4af8-83ec-11e97af6772a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:44.426791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-228e0bf5-ae7f-4af8-83ec-11e97af6772a tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.081315 Jun 10 13:32:44.427360 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1437/2351] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:44 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:45.192016 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-90314d80-c7de-48b0-899f-19dce19aef7c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:45.192619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-90314d80-c7de-48b0-899f-19dce19aef7c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:45.199225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90314d80-c7de-48b0-899f-19dce19aef7c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:45.199676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90314d80-c7de-48b0-899f-19dce19aef7c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:45.200428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90314d80-c7de-48b0-899f-19dce19aef7c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:45.200892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90314d80-c7de-48b0-899f-19dce19aef7c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:45.326666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-90314d80-c7de-48b0-899f-19dce19aef7c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:45.334401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-90314d80-c7de-48b0-899f-19dce19aef7c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1558 microversion: 2.1 time: 0.146208 Jun 10 13:32:45.335058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1438/2352] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:45 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1558 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:45.351041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fcaa1cd8-310e-4951-8d30-b949c980e13b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:45.351596 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fcaa1cd8-310e-4951-8d30-b949c980e13b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:45.357834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fcaa1cd8-310e-4951-8d30-b949c980e13b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:45.363077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fcaa1cd8-310e-4951-8d30-b949c980e13b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:45.363077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fcaa1cd8-310e-4951-8d30-b949c980e13b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:45.363077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fcaa1cd8-310e-4951-8d30-b949c980e13b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:45.438845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fcaa1cd8-310e-4951-8d30-b949c980e13b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.091367 Jun 10 13:32:45.439437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1439/2353] 10.222.0.25 () {58 vars in 1274 bytes} [Fri Jun 10 13:32:45 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7/os-interface => generated 285 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:32:45.443910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1d4744b4-1136-4a53-aff2-ddd39f639b5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:45.444504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1d4744b4-1136-4a53-aff2-ddd39f639b5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:45.451094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d4744b4-1136-4a53-aff2-ddd39f639b5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:45.451531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d4744b4-1136-4a53-aff2-ddd39f639b5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:45.452279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d4744b4-1136-4a53-aff2-ddd39f639b5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:45.452707 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d4744b4-1136-4a53-aff2-ddd39f639b5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:45.545833 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1d4744b4-1136-4a53-aff2-ddd39f639b5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:45.553698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1d4744b4-1136-4a53-aff2-ddd39f639b5f tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.113295 Jun 10 13:32:45.554270 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1440/2354] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:45 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:32:46.389694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-cdc583e8-a1f2-451a-86b4-2242770ae18d req-8840891e-1059-4a21-a1c0-e785cc5c2a7b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b923ce4b-4493-492c-82b8-4adc45ea47f7", "tag": "f237b5cf-c7a4-49a8-b20d-f2f6c15cb9af"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:32:46.398280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-cdc583e8-a1f2-451a-86b4-2242770ae18d req-8840891e-1059-4a21-a1c0-e785cc5c2a7b service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:46.398822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-cdc583e8-a1f2-451a-86b4-2242770ae18d req-8840891e-1059-4a21-a1c0-e785cc5c2a7b service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:46.410536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-cdc583e8-a1f2-451a-86b4-2242770ae18d req-8840891e-1059-4a21-a1c0-e785cc5c2a7b service nova] Creating event network-changed:f237b5cf-c7a4-49a8-b20d-f2f6c15cb9af for instance b923ce4b-4493-492c-82b8-4adc45ea47f7 on hv-cinder-maste Jun 10 13:32:46.416689 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-cdc583e8-a1f2-451a-86b4-2242770ae18d req-8840891e-1059-4a21-a1c0-e785cc5c2a7b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030593 Jun 10 13:32:46.417389 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1441/2355] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:32:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:32:46.571127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-88969435-d058-4f1e-ad27-c9617d7324ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:46.572712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-88969435-d058-4f1e-ad27-c9617d7324ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:46.579478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-88969435-d058-4f1e-ad27-c9617d7324ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:46.579916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-88969435-d058-4f1e-ad27-c9617d7324ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:46.580670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-88969435-d058-4f1e-ad27-c9617d7324ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:46.581120 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-88969435-d058-4f1e-ad27-c9617d7324ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:46.649662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-88969435-d058-4f1e-ad27-c9617d7324ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:46.658926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-88969435-d058-4f1e-ad27-c9617d7324ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.091971 Jun 10 13:32:46.659721 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1442/2356] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:46 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:47.675327 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8b05d395-1260-485c-ab30-5403f42e0f80 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:47.676016 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8b05d395-1260-485c-ab30-5403f42e0f80 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:47.682345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8b05d395-1260-485c-ab30-5403f42e0f80 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:47.682783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8b05d395-1260-485c-ab30-5403f42e0f80 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:47.683523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8b05d395-1260-485c-ab30-5403f42e0f80 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:47.683952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8b05d395-1260-485c-ab30-5403f42e0f80 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:47.747633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8b05d395-1260-485c-ab30-5403f42e0f80 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:47.755854 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8b05d395-1260-485c-ab30-5403f42e0f80 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.084324 Jun 10 13:32:47.756471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1443/2357] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:47 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:48.772939 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8042ffe9-d6fb-47a7-baa6-2daa068905dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:48.773662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8042ffe9-d6fb-47a7-baa6-2daa068905dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:48.776691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:32:48.781409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8042ffe9-d6fb-47a7-baa6-2daa068905dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:48.781856 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8042ffe9-d6fb-47a7-baa6-2daa068905dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:48.782614 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8042ffe9-d6fb-47a7-baa6-2daa068905dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:48.783046 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8042ffe9-d6fb-47a7-baa6-2daa068905dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:48.844736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8042ffe9-d6fb-47a7-baa6-2daa068905dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:48.852662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8042ffe9-d6fb-47a7-baa6-2daa068905dd tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.083693 Jun 10 13:32:48.853256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1444/2358] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:48 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:49.869622 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-75236a9e-0b4e-42ea-9fc8-d0b75334dfe3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:49.870245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-75236a9e-0b4e-42ea-9fc8-d0b75334dfe3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:49.876963 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75236a9e-0b4e-42ea-9fc8-d0b75334dfe3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:49.877407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75236a9e-0b4e-42ea-9fc8-d0b75334dfe3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:49.878160 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75236a9e-0b4e-42ea-9fc8-d0b75334dfe3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:49.878591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75236a9e-0b4e-42ea-9fc8-d0b75334dfe3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:49.937827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-75236a9e-0b4e-42ea-9fc8-d0b75334dfe3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:49.946626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-75236a9e-0b4e-42ea-9fc8-d0b75334dfe3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.080371 Jun 10 13:32:49.947196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1445/2359] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:49 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:50.963711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-128eb7a2-1c84-4c58-8a98-8dd99f08ef07 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:50.964358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-128eb7a2-1c84-4c58-8a98-8dd99f08ef07 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:50.970792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-128eb7a2-1c84-4c58-8a98-8dd99f08ef07 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:50.971234 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-128eb7a2-1c84-4c58-8a98-8dd99f08ef07 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:50.971977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-128eb7a2-1c84-4c58-8a98-8dd99f08ef07 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:50.972404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-128eb7a2-1c84-4c58-8a98-8dd99f08ef07 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:50.976322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:32:51.035440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-128eb7a2-1c84-4c58-8a98-8dd99f08ef07 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:51.043288 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-128eb7a2-1c84-4c58-8a98-8dd99f08ef07 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.083681 Jun 10 13:32:51.043845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1446/2360] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:50 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:52.060734 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2e06e465-b95f-4f7c-bd9e-d4a854b5ca88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:52.061396 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2e06e465-b95f-4f7c-bd9e-d4a854b5ca88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:52.067764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e06e465-b95f-4f7c-bd9e-d4a854b5ca88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:52.068207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e06e465-b95f-4f7c-bd9e-d4a854b5ca88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:52.068979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e06e465-b95f-4f7c-bd9e-d4a854b5ca88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:52.069404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e06e465-b95f-4f7c-bd9e-d4a854b5ca88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:52.129255 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2e06e465-b95f-4f7c-bd9e-d4a854b5ca88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:52.137190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2e06e465-b95f-4f7c-bd9e-d4a854b5ca88 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.080294 Jun 10 13:32:52.137758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1447/2361] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:52 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:53.153755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e8a205f0-8964-46a7-a294-de79509a030b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:53.154399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e8a205f0-8964-46a7-a294-de79509a030b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:53.160921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8a205f0-8964-46a7-a294-de79509a030b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:53.161362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8a205f0-8964-46a7-a294-de79509a030b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:53.162115 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8a205f0-8964-46a7-a294-de79509a030b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:53.162656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e8a205f0-8964-46a7-a294-de79509a030b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:53.222333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e8a205f0-8964-46a7-a294-de79509a030b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:53.230206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e8a205f0-8964-46a7-a294-de79509a030b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.080324 Jun 10 13:32:53.230764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1448/2362] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:53 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:54.247896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5bdfd730-8d05-4266-8369-005d32d76ea3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:54.248562 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5bdfd730-8d05-4266-8369-005d32d76ea3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:54.254996 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5bdfd730-8d05-4266-8369-005d32d76ea3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:54.255442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5bdfd730-8d05-4266-8369-005d32d76ea3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:54.256204 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5bdfd730-8d05-4266-8369-005d32d76ea3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:54.256635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5bdfd730-8d05-4266-8369-005d32d76ea3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:54.317106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5bdfd730-8d05-4266-8369-005d32d76ea3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:54.325018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5bdfd730-8d05-4266-8369-005d32d76ea3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.081721 Jun 10 13:32:54.325574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1449/2363] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:54 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:55.342142 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8ee8a728-7b82-463f-bd0f-355317f30227 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:55.342752 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8ee8a728-7b82-463f-bd0f-355317f30227 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:55.349349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ee8a728-7b82-463f-bd0f-355317f30227 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:55.349797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ee8a728-7b82-463f-bd0f-355317f30227 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:55.350551 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ee8a728-7b82-463f-bd0f-355317f30227 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:55.350986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ee8a728-7b82-463f-bd0f-355317f30227 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:55.414381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8ee8a728-7b82-463f-bd0f-355317f30227 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:55.422262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8ee8a728-7b82-463f-bd0f-355317f30227 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.083988 Jun 10 13:32:55.422832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1450/2364] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:55 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:56.439715 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ac0052a5-bb7d-44e3-9cbc-306b3efeb763 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:56.440357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ac0052a5-bb7d-44e3-9cbc-306b3efeb763 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:56.447263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ac0052a5-bb7d-44e3-9cbc-306b3efeb763 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:56.447705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ac0052a5-bb7d-44e3-9cbc-306b3efeb763 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:56.448460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ac0052a5-bb7d-44e3-9cbc-306b3efeb763 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:56.448928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ac0052a5-bb7d-44e3-9cbc-306b3efeb763 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:56.507893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ac0052a5-bb7d-44e3-9cbc-306b3efeb763 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:56.516353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ac0052a5-bb7d-44e3-9cbc-306b3efeb763 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.080578 Jun 10 13:32:56.516938 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1451/2365] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:56 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:57.533242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2348a577-792c-4ea3-abaa-a50b1b9615a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:57.533889 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2348a577-792c-4ea3-abaa-a50b1b9615a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:57.540300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2348a577-792c-4ea3-abaa-a50b1b9615a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:57.540822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2348a577-792c-4ea3-abaa-a50b1b9615a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:57.547089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2348a577-792c-4ea3-abaa-a50b1b9615a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:57.547089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2348a577-792c-4ea3-abaa-a50b1b9615a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:57.601210 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2348a577-792c-4ea3-abaa-a50b1b9615a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:57.609073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2348a577-792c-4ea3-abaa-a50b1b9615a5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.079958 Jun 10 13:32:57.609635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1452/2366] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:57 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:32:58.625950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a4d4fb10-3421-4fff-acbd-0297173175ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:58.626608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a4d4fb10-3421-4fff-acbd-0297173175ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:58.632875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a4d4fb10-3421-4fff-acbd-0297173175ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:58.633347 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a4d4fb10-3421-4fff-acbd-0297173175ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:58.634093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a4d4fb10-3421-4fff-acbd-0297173175ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:58.634518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a4d4fb10-3421-4fff-acbd-0297173175ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:58.692826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a4d4fb10-3421-4fff-acbd-0297173175ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:58.700597 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a4d4fb10-3421-4fff-acbd-0297173175ab tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.078589 Jun 10 13:32:58.701178 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1453/2367] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:58 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:32:59.717735 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cb3dc776-efb8-4ffa-89b5-76b9eb77d778 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:32:59.718397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cb3dc776-efb8-4ffa-89b5-76b9eb77d778 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:32:59.725218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb3dc776-efb8-4ffa-89b5-76b9eb77d778 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:59.725658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb3dc776-efb8-4ffa-89b5-76b9eb77d778 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:59.726685 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb3dc776-efb8-4ffa-89b5-76b9eb77d778 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:32:59.727188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb3dc776-efb8-4ffa-89b5-76b9eb77d778 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:32:59.789169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cb3dc776-efb8-4ffa-89b5-76b9eb77d778 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:32:59.796986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cb3dc776-efb8-4ffa-89b5-76b9eb77d778 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.083041 Jun 10 13:32:59.797548 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1454/2368] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:32:59 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:00.814302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1d0cfce4-c31d-422f-8371-fad362b1d1eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:00.814974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1d0cfce4-c31d-422f-8371-fad362b1d1eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:00.821903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d0cfce4-c31d-422f-8371-fad362b1d1eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:00.822345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d0cfce4-c31d-422f-8371-fad362b1d1eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:00.823086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d0cfce4-c31d-422f-8371-fad362b1d1eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:00.823512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1d0cfce4-c31d-422f-8371-fad362b1d1eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:00.882103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1d0cfce4-c31d-422f-8371-fad362b1d1eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:00.890440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1d0cfce4-c31d-422f-8371-fad362b1d1eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.080017 Jun 10 13:33:00.890995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1455/2369] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:00 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:01.907733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c2d739a6-2a41-46c6-9bfb-12bd7cfad583 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:01.908383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c2d739a6-2a41-46c6-9bfb-12bd7cfad583 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:01.914947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c2d739a6-2a41-46c6-9bfb-12bd7cfad583 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:01.915820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c2d739a6-2a41-46c6-9bfb-12bd7cfad583 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:01.916202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c2d739a6-2a41-46c6-9bfb-12bd7cfad583 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:01.916639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c2d739a6-2a41-46c6-9bfb-12bd7cfad583 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:01.979154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c2d739a6-2a41-46c6-9bfb-12bd7cfad583 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:01.986957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c2d739a6-2a41-46c6-9bfb-12bd7cfad583 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.083261 Jun 10 13:33:01.987521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1456/2370] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:01 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:03.004086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9cf5ee08-a56b-4f55-8216-208b3f99cdb7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:03.004725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9cf5ee08-a56b-4f55-8216-208b3f99cdb7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:03.011897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cf5ee08-a56b-4f55-8216-208b3f99cdb7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:03.012342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cf5ee08-a56b-4f55-8216-208b3f99cdb7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:03.013133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cf5ee08-a56b-4f55-8216-208b3f99cdb7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:03.013569 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9cf5ee08-a56b-4f55-8216-208b3f99cdb7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:03.076147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9cf5ee08-a56b-4f55-8216-208b3f99cdb7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:03.080547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9cf5ee08-a56b-4f55-8216-208b3f99cdb7 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.080324 Jun 10 13:33:03.083463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1457/2371] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:02 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:04.097547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-253f967b-b032-467c-865d-88ab28c31c81 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:04.098217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-253f967b-b032-467c-865d-88ab28c31c81 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:04.104730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-253f967b-b032-467c-865d-88ab28c31c81 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:04.105196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-253f967b-b032-467c-865d-88ab28c31c81 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:04.105937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-253f967b-b032-467c-865d-88ab28c31c81 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:04.106364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-253f967b-b032-467c-865d-88ab28c31c81 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:04.169575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-253f967b-b032-467c-865d-88ab28c31c81 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:04.177456 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-253f967b-b032-467c-865d-88ab28c31c81 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.083726 Jun 10 13:33:04.178599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1458/2372] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:04 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:05.195846 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7afa6827-47e6-4113-92c7-12ff26ccf51c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:05.196497 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7afa6827-47e6-4113-92c7-12ff26ccf51c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:05.203348 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7afa6827-47e6-4113-92c7-12ff26ccf51c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:05.203796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7afa6827-47e6-4113-92c7-12ff26ccf51c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:05.204557 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7afa6827-47e6-4113-92c7-12ff26ccf51c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:05.205030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7afa6827-47e6-4113-92c7-12ff26ccf51c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:05.267061 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7afa6827-47e6-4113-92c7-12ff26ccf51c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:05.274843 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7afa6827-47e6-4113-92c7-12ff26ccf51c tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.082837 Jun 10 13:33:05.275903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1459/2373] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:05 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:06.291442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5912aaf8-dbd2-4922-8d85-0357d7b4c8d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:06.292078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5912aaf8-dbd2-4922-8d85-0357d7b4c8d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:06.300791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5912aaf8-dbd2-4922-8d85-0357d7b4c8d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:06.301359 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5912aaf8-dbd2-4922-8d85-0357d7b4c8d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:06.302155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5912aaf8-dbd2-4922-8d85-0357d7b4c8d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:06.302585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5912aaf8-dbd2-4922-8d85-0357d7b4c8d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:06.362550 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5912aaf8-dbd2-4922-8d85-0357d7b4c8d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:06.371524 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5912aaf8-dbd2-4922-8d85-0357d7b4c8d3 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.083946 Jun 10 13:33:06.372091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1460/2374] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:06 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:07.387381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9ef512e8-0a0b-456a-b85c-5935b5dbf7a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:07.387991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9ef512e8-0a0b-456a-b85c-5935b5dbf7a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:07.394399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ef512e8-0a0b-456a-b85c-5935b5dbf7a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:07.394841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ef512e8-0a0b-456a-b85c-5935b5dbf7a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:07.395585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ef512e8-0a0b-456a-b85c-5935b5dbf7a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:07.396008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9ef512e8-0a0b-456a-b85c-5935b5dbf7a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:07.456608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9ef512e8-0a0b-456a-b85c-5935b5dbf7a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:07.464937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9ef512e8-0a0b-456a-b85c-5935b5dbf7a9 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.081377 Jun 10 13:33:07.465504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1461/2375] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:07 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:08.487516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e056a27d-7181-488e-8647-4baacbd5f5ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:08.488148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e056a27d-7181-488e-8647-4baacbd5f5ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:08.495295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e056a27d-7181-488e-8647-4baacbd5f5ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:08.496507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e056a27d-7181-488e-8647-4baacbd5f5ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:08.496507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e056a27d-7181-488e-8647-4baacbd5f5ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:08.497040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e056a27d-7181-488e-8647-4baacbd5f5ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:08.559110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e056a27d-7181-488e-8647-4baacbd5f5ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:08.562491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:33:08.567246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e056a27d-7181-488e-8647-4baacbd5f5ae tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.083697 Jun 10 13:33:08.567803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1462/2376] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:08 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:09.584724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ba11b9d2-b744-4652-93ca-2928bfb08664 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:09.585390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ba11b9d2-b744-4652-93ca-2928bfb08664 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:09.591914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba11b9d2-b744-4652-93ca-2928bfb08664 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:09.592307 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba11b9d2-b744-4652-93ca-2928bfb08664 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:09.593098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba11b9d2-b744-4652-93ca-2928bfb08664 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:09.593531 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ba11b9d2-b744-4652-93ca-2928bfb08664 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:09.669943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ba11b9d2-b744-4652-93ca-2928bfb08664 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:09.670737 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ba11b9d2-b744-4652-93ca-2928bfb08664 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.090213 Jun 10 13:33:09.671305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1463/2377] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:09 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:10.688168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-771f8b48-2c13-45d5-bd8a-244f7d562918 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:10.688783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-771f8b48-2c13-45d5-bd8a-244f7d562918 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:10.695175 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-771f8b48-2c13-45d5-bd8a-244f7d562918 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:10.695612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-771f8b48-2c13-45d5-bd8a-244f7d562918 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:10.696387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-771f8b48-2c13-45d5-bd8a-244f7d562918 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:10.696836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-771f8b48-2c13-45d5-bd8a-244f7d562918 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:10.755788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-771f8b48-2c13-45d5-bd8a-244f7d562918 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:10.763529 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-771f8b48-2c13-45d5-bd8a-244f7d562918 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.079173 Jun 10 13:33:10.764082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1464/2378] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:10 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:11.779469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a8542a81-d7b1-4cad-bb4a-54136fb661eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:11.780158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a8542a81-d7b1-4cad-bb4a-54136fb661eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:11.786607 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8542a81-d7b1-4cad-bb4a-54136fb661eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:11.787058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8542a81-d7b1-4cad-bb4a-54136fb661eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:11.787813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8542a81-d7b1-4cad-bb4a-54136fb661eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:11.788314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8542a81-d7b1-4cad-bb4a-54136fb661eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:11.847486 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a8542a81-d7b1-4cad-bb4a-54136fb661eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:11.855512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a8542a81-d7b1-4cad-bb4a-54136fb661eb tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.079882 Jun 10 13:33:11.864239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1465/2379] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:11 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:12.872499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-626d1608-9d39-4718-a0fa-26e685b6a3ee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:12.873184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-626d1608-9d39-4718-a0fa-26e685b6a3ee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:12.879651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-626d1608-9d39-4718-a0fa-26e685b6a3ee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:12.880098 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-626d1608-9d39-4718-a0fa-26e685b6a3ee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:12.880885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-626d1608-9d39-4718-a0fa-26e685b6a3ee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:12.881323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-626d1608-9d39-4718-a0fa-26e685b6a3ee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:12.955587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-626d1608-9d39-4718-a0fa-26e685b6a3ee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:12.964174 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-626d1608-9d39-4718-a0fa-26e685b6a3ee tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.095559 Jun 10 13:33:12.964737 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1466/2380] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:12 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:13.981655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-95aaac2d-add0-45fd-be5c-fe3629b8573b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:13.982326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-95aaac2d-add0-45fd-be5c-fe3629b8573b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:13.988724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95aaac2d-add0-45fd-be5c-fe3629b8573b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:13.989211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95aaac2d-add0-45fd-be5c-fe3629b8573b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:13.989964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95aaac2d-add0-45fd-be5c-fe3629b8573b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:13.990394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95aaac2d-add0-45fd-be5c-fe3629b8573b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:14.049751 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-95aaac2d-add0-45fd-be5c-fe3629b8573b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:14.057507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-95aaac2d-add0-45fd-be5c-fe3629b8573b tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.080531 Jun 10 13:33:14.058069 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1467/2381] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:13 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:15.074298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a99fcc4b-5fc0-4221-b183-1f63d86acada tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:15.074961 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a99fcc4b-5fc0-4221-b183-1f63d86acada tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:15.081747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a99fcc4b-5fc0-4221-b183-1f63d86acada tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:15.082186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a99fcc4b-5fc0-4221-b183-1f63d86acada tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:15.082932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a99fcc4b-5fc0-4221-b183-1f63d86acada tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:15.083356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a99fcc4b-5fc0-4221-b183-1f63d86acada tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:15.141828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a99fcc4b-5fc0-4221-b183-1f63d86acada tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:15.149457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a99fcc4b-5fc0-4221-b183-1f63d86acada tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.079009 Jun 10 13:33:15.150013 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1468/2382] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:15 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:16.165559 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8533fdcf-1680-486d-9e09-f8a5755d5cfc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:16.166228 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8533fdcf-1680-486d-9e09-f8a5755d5cfc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:16.172608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8533fdcf-1680-486d-9e09-f8a5755d5cfc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:16.173079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8533fdcf-1680-486d-9e09-f8a5755d5cfc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:16.173827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8533fdcf-1680-486d-9e09-f8a5755d5cfc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:16.174256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8533fdcf-1680-486d-9e09-f8a5755d5cfc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:16.235442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8533fdcf-1680-486d-9e09-f8a5755d5cfc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:16.243148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8533fdcf-1680-486d-9e09-f8a5755d5cfc tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.081459 Jun 10 13:33:16.244230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1469/2383] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:16 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:17.260399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c5d1d9c6-700b-456b-b951-a77caf2753e2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:17.261135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c5d1d9c6-700b-456b-b951-a77caf2753e2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:17.267623 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5d1d9c6-700b-456b-b951-a77caf2753e2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:17.269578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5d1d9c6-700b-456b-b951-a77caf2753e2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:17.269578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5d1d9c6-700b-456b-b951-a77caf2753e2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:17.269578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5d1d9c6-700b-456b-b951-a77caf2753e2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:17.332512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c5d1d9c6-700b-456b-b951-a77caf2753e2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:17.340643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c5d1d9c6-700b-456b-b951-a77caf2753e2 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.084673 Jun 10 13:33:17.341257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1470/2384] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:17 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:18.356860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-083d6d52-1c5f-4506-9897-b39c7781c283 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:18.357494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-083d6d52-1c5f-4506-9897-b39c7781c283 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:18.364337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-083d6d52-1c5f-4506-9897-b39c7781c283 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:18.364830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-083d6d52-1c5f-4506-9897-b39c7781c283 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:18.365591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-083d6d52-1c5f-4506-9897-b39c7781c283 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:18.366020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-083d6d52-1c5f-4506-9897-b39c7781c283 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:18.424528 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-083d6d52-1c5f-4506-9897-b39c7781c283 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:18.433001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-083d6d52-1c5f-4506-9897-b39c7781c283 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.080004 Jun 10 13:33:18.433859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1471/2385] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:18 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:19.450108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5cf5fcee-0f46-48a1-91d6-5c39fecb80b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:19.450752 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5cf5fcee-0f46-48a1-91d6-5c39fecb80b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:19.457530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5cf5fcee-0f46-48a1-91d6-5c39fecb80b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:19.457986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5cf5fcee-0f46-48a1-91d6-5c39fecb80b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:19.458749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5cf5fcee-0f46-48a1-91d6-5c39fecb80b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:19.459191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5cf5fcee-0f46-48a1-91d6-5c39fecb80b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:19.518838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5cf5fcee-0f46-48a1-91d6-5c39fecb80b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:19.527386 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5cf5fcee-0f46-48a1-91d6-5c39fecb80b5 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.081308 Jun 10 13:33:19.527964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1472/2386] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:19 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:20.544557 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-acb6e9b3-5458-44c4-b47b-74f8a620d334 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:20.545247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-acb6e9b3-5458-44c4-b47b-74f8a620d334 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:20.547446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=58 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:33:20.552123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acb6e9b3-5458-44c4-b47b-74f8a620d334 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:20.552561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acb6e9b3-5458-44c4-b47b-74f8a620d334 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:20.553344 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acb6e9b3-5458-44c4-b47b-74f8a620d334 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:20.553768 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acb6e9b3-5458-44c4-b47b-74f8a620d334 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:20.613211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-acb6e9b3-5458-44c4-b47b-74f8a620d334 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:20.621402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-acb6e9b3-5458-44c4-b47b-74f8a620d334 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.080721 Jun 10 13:33:20.621965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1473/2387] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:20 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:21.638522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5f5e122e-3228-4955-bc53-bb431761d031 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:21.639152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5f5e122e-3228-4955-bc53-bb431761d031 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] [instance: a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:21.645478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5f5e122e-3228-4955-bc53-bb431761d031 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:21.645921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5f5e122e-3228-4955-bc53-bb431761d031 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:21.646668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5f5e122e-3228-4955-bc53-bb431761d031 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:21.647095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5f5e122e-3228-4955-bc53-bb431761d031 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:21.708286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5f5e122e-3228-4955-bc53-bb431761d031 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '849e36f8653d4d1f9a7fb1df8fb913a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff2616337ae440a89047fd9a50218387', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:33:21.716134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5f5e122e-3228-4955-bc53-bb431761d031 tempest-VolumesSnapshotTestJSON-1698707508 tempest-VolumesSnapshotTestJSON-1698707508-project] 10.222.0.25 "GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744" status: 200 len: 1349 microversion: 2.1 time: 0.082069 Jun 10 13:33:21.716697 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1474/2388] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:33:21 2022] GET /compute/v2.1/servers/a32f3e1f-06eb-43ce-9dfb-dbae7d4aa744 => generated 1349 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:33:22.412861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-956d01db-7ed9-4bbe-bad4-d5e7d2d2ba72 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:33:22.413344 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-956d01db-7ed9-4bbe-bad4-d5e7d2d2ba72 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:33:22.419340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-956d01db-7ed9-4bbe-bad4-d5e7d2d2ba72 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:22.419779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-956d01db-7ed9-4bbe-bad4-d5e7d2d2ba72 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:22.420517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-956d01db-7ed9-4bbe-bad4-d5e7d2d2ba72 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:33:22.420971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-956d01db-7ed9-4bbe-bad4-d5e7d2d2ba72 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:33:22.442997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-956d01db-7ed9-4bbe-bad4-d5e7d2d2ba72 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7/ips" status: 200 len: 150 microversion: 2.1 time: 0.034028 Jun 10 13:33:22.443556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1475/2389] 10.222.0.25 () {58 vars in 1247 bytes} [Fri Jun 10 13:33:22 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7/ips => generated 150 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:35:14.217134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:35:14.217856 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:35:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:35:35.248808 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:35:35.249440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:35:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:36:17.963048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR oslo.messaging._drivers.impl_rabbit [-] [28f4d62e-58f2-4fc0-963e-d17379b93d6e] AMQP server on 10.222.0.25:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Jun 10 13:36:17.963987 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=5 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:36:17.964900 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=76 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:36:17.968865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-184ee3c0-ee03-4304-b85b-7499ad52afe2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:36:17.969364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-184ee3c0-ee03-4304-b85b-7499ad52afe2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:36:17.984080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-184ee3c0-ee03-4304-b85b-7499ad52afe2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:36:17.984080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-184ee3c0-ee03-4304-b85b-7499ad52afe2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:36:17.984080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-184ee3c0-ee03-4304-b85b-7499ad52afe2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:36:17.984080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-184ee3c0-ee03-4304-b85b-7499ad52afe2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:36:18.013749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-184ee3c0-ee03-4304-b85b-7499ad52afe2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:36:18.309043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-184ee3c0-ee03-4304-b85b-7499ad52afe2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "DELETE /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 204 len: 0 microversion: 2.20 time: 0.347461 Jun 10 13:36:18.309553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1476/2390] 10.222.0.25 () {60 vars in 1279 bytes} [Fri Jun 10 13:36:17 2022] DELETE /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Jun 10 13:36:18.318262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1c00e0f5-39b9-4da1-9601-d6cea92f6796 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:36:18.319493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1c00e0f5-39b9-4da1-9601-d6cea92f6796 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:36:18.325578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c00e0f5-39b9-4da1-9601-d6cea92f6796 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:36:18.331879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c00e0f5-39b9-4da1-9601-d6cea92f6796 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:36:18.331879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c00e0f5-39b9-4da1-9601-d6cea92f6796 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:36:18.331879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c00e0f5-39b9-4da1-9601-d6cea92f6796 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:36:18.504304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1c00e0f5-39b9-4da1-9601-d6cea92f6796 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:36:18.512479 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1c00e0f5-39b9-4da1-9601-d6cea92f6796 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:36:18.514091 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1c00e0f5-39b9-4da1-9601-d6cea92f6796 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:36:18.515249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1c00e0f5-39b9-4da1-9601-d6cea92f6796 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1721 microversion: 2.20 time: 0.201436 Jun 10 13:36:18.515802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1477/2391] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:36:18 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1721 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:36:19.534602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-79c15d38-7676-43bf-9e6d-8a823f45779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:36:19.535256 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-79c15d38-7676-43bf-9e6d-8a823f45779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:36:19.544826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79c15d38-7676-43bf-9e6d-8a823f45779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:36:19.545275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79c15d38-7676-43bf-9e6d-8a823f45779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:36:19.546047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79c15d38-7676-43bf-9e6d-8a823f45779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:36:19.546475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-79c15d38-7676-43bf-9e6d-8a823f45779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:36:19.554481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] [28f4d62e-58f2-4fc0-963e-d17379b93d6e] Reconnected to AMQP server on 10.222.0.25:5672 via [amqp] client with port 42046. Jun 10 13:36:19.676965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-79c15d38-7676-43bf-9e6d-8a823f45779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:36:19.685275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-79c15d38-7676-43bf-9e6d-8a823f45779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:36:19.686724 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-79c15d38-7676-43bf-9e6d-8a823f45779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:36:19.687878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-79c15d38-7676-43bf-9e6d-8a823f45779b tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1721 microversion: 2.20 time: 0.157799 Jun 10 13:36:19.688435 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1478/2392] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:36:19 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1721 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:36:20.136243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-2ec702df-55b9-41db-b95f-3cdaf7a6f1e4 req-540c6930-ff09-42d9-bebc-d856a5766597 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a99c1a73-3f56-4c71-a4fd-67988bbb7464", "name": "network-vif-deleted", "tag": "a5d10747-5fee-4e44-ae6e-53bfbd5134eb"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:36:20.142079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-2ec702df-55b9-41db-b95f-3cdaf7a6f1e4 req-540c6930-ff09-42d9-bebc-d856a5766597 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:36:20.143332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-2ec702df-55b9-41db-b95f-3cdaf7a6f1e4 req-540c6930-ff09-42d9-bebc-d856a5766597 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:36:20.155571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-2ec702df-55b9-41db-b95f-3cdaf7a6f1e4 req-540c6930-ff09-42d9-bebc-d856a5766597 service nova] Creating event network-vif-deleted:a5d10747-5fee-4e44-ae6e-53bfbd5134eb for instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 on hv-cinder-maste Jun 10 13:36:20.161791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-2ec702df-55b9-41db-b95f-3cdaf7a6f1e4 req-540c6930-ff09-42d9-bebc-d856a5766597 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.104479 Jun 10 13:36:20.162870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1479/2393] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:36:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:36:20.707629 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ab9f0971-2a7a-401f-80bc-d06b976bb73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:36:20.708372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ab9f0971-2a7a-401f-80bc-d06b976bb73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:36:20.714710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ab9f0971-2a7a-401f-80bc-d06b976bb73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:36:20.715153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ab9f0971-2a7a-401f-80bc-d06b976bb73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:36:20.715920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ab9f0971-2a7a-401f-80bc-d06b976bb73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:36:20.716343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ab9f0971-2a7a-401f-80bc-d06b976bb73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:36:20.747418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ab9f0971-2a7a-401f-80bc-d06b976bb73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:36:20.747863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ab9f0971-2a7a-401f-80bc-d06b976bb73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:36:20.748305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-ab9f0971-2a7a-401f-80bc-d06b976bb73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lazy-loading 'fault' on Instance uuid a99c1a73-3f56-4c71-a4fd-67988bbb7464 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:36:20.792653 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ab9f0971-2a7a-401f-80bc-d06b976bb73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:36:20.801282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ab9f0971-2a7a-401f-80bc-d06b976bb73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:36:20.802771 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ab9f0971-2a7a-401f-80bc-d06b976bb73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b1d1712ea523498f92f258383863b906', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28f8244decff453e98e2d372a97a563c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:36:20.803916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ab9f0971-2a7a-401f-80bc-d06b976bb73e tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 200 len: 1369 microversion: 2.20 time: 0.101187 Jun 10 13:36:20.804471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1480/2394] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:36:20 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 1369 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jun 10 13:36:21.822278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e4d0ba5b-a5b3-4759-8f5e-229ef120b62d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:36:21.822878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e4d0ba5b-a5b3-4759-8f5e-229ef120b62d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:36:21.829297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4d0ba5b-a5b3-4759-8f5e-229ef120b62d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:36:21.842844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4d0ba5b-a5b3-4759-8f5e-229ef120b62d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:36:21.842844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4d0ba5b-a5b3-4759-8f5e-229ef120b62d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:36:21.842844 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4d0ba5b-a5b3-4759-8f5e-229ef120b62d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:36:21.844574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-e4d0ba5b-a5b3-4759-8f5e-229ef120b62d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] HTTP exception thrown: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e4d0ba5b-a5b3-4759-8f5e-229ef120b62d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Request method failure captured: Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 HTTP/1.1 Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0GI4neGqGl6u4NtxR88mPQrCvR_v1VmiIEaXPKEqlwdyaT_vIeeEJpWEM0993X6W9IucjryiBRG6JJNHEx9ulNG1lmVs69VgCwa572eOjYLN6_EoRIIXPkdDj4Zac85isYH3jNfGblme5mU9zzCrvjMFkL5ny73zNQIxIWZatGPexU Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Openstack-Nova-Api-Version: 2.20 Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 28f8244decff453e98e2d372a97a563c Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:36:21.846323 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:36:21.848257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/28f8244decff453e98e2d372a97a563c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/28f8244decff453e98e2d372a97a563c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_28f8244decff453e98e2d372a97a563c", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/28f8244decff453e98e2d372a97a563c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:36:21.848257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:36:21.848257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 28f8244decff453e98e2d372a97a563c Jun 10 13:36:21.848257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:36:21.848257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeShelveTestJSON-1465745724-project Jun 10 13:36:21.848257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:36:21.848257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:36:21.848257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: b1d1712ea523498f92f258383863b906 Jun 10 13:36:21.848257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1465745724-project Jun 10 13:36:21.848257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:36:21.848257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:36:21.848257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'a99c1a73-3f56-4c71-a4fd-67988bbb7464'} Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:36:21.849402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:36:21.851222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:36:21.852884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e4d0ba5b-a5b3-4759-8f5e-229ef120b62d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Returning 404 to user: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:36:21.852884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e4d0ba5b-a5b3-4759-8f5e-229ef120b62d tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 404 len: 111 microversion: 2.20 time: 0.029608 Jun 10 13:36:21.852884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1481/2395] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:36:21 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jun 10 13:36:21.856111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9515fb07-ff1e-43f3-92df-25df318a2ec2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '.version_select at 0x7f9309921550>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:36:21.949705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9515fb07-ff1e-43f3-92df-25df318a2ec2 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-224439963" status: 204 len: 0 microversion: 2.20 time: 0.097070 Jun 10 13:36:21.950273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1482/2396] 10.222.0.25 () {60 vars in 1258 bytes} [Fri Jun 10 13:36:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-224439963 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Jun 10 13:37:22.863583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=38,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:37:22.864402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=22,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:37:22.868507 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b24cf763-a95a-47ac-8037-f4cc673f0900 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:37:22.869022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b24cf763-a95a-47ac-8037-f4cc673f0900 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:37:22.875546 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b24cf763-a95a-47ac-8037-f4cc673f0900 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:22.882613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b24cf763-a95a-47ac-8037-f4cc673f0900 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:22.882613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b24cf763-a95a-47ac-8037-f4cc673f0900 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:22.882613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b24cf763-a95a-47ac-8037-f4cc673f0900 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:22.901163 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b24cf763-a95a-47ac-8037-f4cc673f0900 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:37:23.206038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b24cf763-a95a-47ac-8037-f4cc673f0900 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "DELETE /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 204 len: 0 microversion: 2.1 time: 0.344114 Jun 10 13:37:23.206633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1483/2397] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:37:22 2022] DELETE /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jun 10 13:37:23.214778 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8a5aced3-4068-4227-bde0-301dd5c168dc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:37:23.215471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8a5aced3-4068-4227-bde0-301dd5c168dc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:37:23.221520 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a5aced3-4068-4227-bde0-301dd5c168dc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:23.221970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a5aced3-4068-4227-bde0-301dd5c168dc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:23.222725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a5aced3-4068-4227-bde0-301dd5c168dc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:23.223156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a5aced3-4068-4227-bde0-301dd5c168dc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:23.537938 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8a5aced3-4068-4227-bde0-301dd5c168dc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:37:23.545326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8a5aced3-4068-4227-bde0-301dd5c168dc tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1508 microversion: 2.1 time: 0.334501 Jun 10 13:37:23.545883 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1484/2398] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:37:23 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1508 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:37:24.563595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f1f880a0-b304-4053-b43d-59c012e27bab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:37:24.564244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f1f880a0-b304-4053-b43d-59c012e27bab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:37:24.571317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1f880a0-b304-4053-b43d-59c012e27bab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:24.571833 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1f880a0-b304-4053-b43d-59c012e27bab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:24.572668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1f880a0-b304-4053-b43d-59c012e27bab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:24.573212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1f880a0-b304-4053-b43d-59c012e27bab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:24.637851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f1f880a0-b304-4053-b43d-59c012e27bab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:37:24.645992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f1f880a0-b304-4053-b43d-59c012e27bab tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 200 len: 1466 microversion: 2.1 time: 0.086533 Jun 10 13:37:24.646565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1485/2399] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:37:24 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 1466 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:37:24.651031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-26e60481-e729-4b43-9881-b15ea933b2d2 req-ca81f9b4-cd6f-477c-9dee-7bcd902ee7e5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6ddb4431-e631-4edd-af56-46cbe0241ca8", "name": "network-vif-deleted", "tag": "ec274b26-171c-4c68-9350-cbe87c6ec7bc"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:37:24.659912 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-26e60481-e729-4b43-9881-b15ea933b2d2 req-ca81f9b4-cd6f-477c-9dee-7bcd902ee7e5 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:24.660352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-26e60481-e729-4b43-9881-b15ea933b2d2 req-ca81f9b4-cd6f-477c-9dee-7bcd902ee7e5 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:24.672915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-26e60481-e729-4b43-9881-b15ea933b2d2 req-ca81f9b4-cd6f-477c-9dee-7bcd902ee7e5 service nova] Creating event network-vif-deleted:ec274b26-171c-4c68-9350-cbe87c6ec7bc for instance 6ddb4431-e631-4edd-af56-46cbe0241ca8 on hv-cinder-maste Jun 10 13:37:24.676369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-26e60481-e729-4b43-9881-b15ea933b2d2 req-ca81f9b4-cd6f-477c-9dee-7bcd902ee7e5 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028588 Jun 10 13:37:24.677188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1486/2400] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:37:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:37:25.663997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b99e8aa4-f5b7-41ea-9bdf-a24b80bef548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:37:25.664671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b99e8aa4-f5b7-41ea-9bdf-a24b80bef548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 6ddb4431-e631-4edd-af56-46cbe0241ca8] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:37:25.670815 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b99e8aa4-f5b7-41ea-9bdf-a24b80bef548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:25.671267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b99e8aa4-f5b7-41ea-9bdf-a24b80bef548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:25.673518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b99e8aa4-f5b7-41ea-9bdf-a24b80bef548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:25.673518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b99e8aa4-f5b7-41ea-9bdf-a24b80bef548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:25.686674 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-b99e8aa4-f5b7-41ea-9bdf-a24b80bef548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] HTTP exception thrown: Instance 6ddb4431-e631-4edd-af56-46cbe0241ca8 could not be found. Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b99e8aa4-f5b7-41ea-9bdf-a24b80bef548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Request method failure captured: Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 HTTP/1.1 Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0HbeUNf0EPZ-aslwQVWNd8-Z8OHNvp8Sb8m5gjvFM0tL0pvFj03ha5wm_h98nRGWdTUfHXBkNJa9XdKTFzGaRbJsRxJn5YRVRoYaqROYy-SpZpzuBlLa9e8KDT13WQC8G8gU0qsOwDmzLx6BmD9PCuaOyoARKjO1sMu6-pWpQAV11w Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 03423e3482614374afa8c929bd931262 Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-696024477 Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:37:25.688346 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:37:25.690202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/03423e3482614374afa8c929bd931262"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/03423e3482614374afa8c929bd931262"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_03423e3482614374afa8c929bd931262", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/03423e3482614374afa8c929bd931262"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:37:25.690202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-ServerRescueNegativeTestJSON-696024477 Jun 10 13:37:25.690202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 03423e3482614374afa8c929bd931262 Jun 10 13:37:25.690202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-696024477 Jun 10 13:37:25.690202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-ServerRescueNegativeTestJSON-696024477-project Jun 10 13:37:25.690202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:37:25.690202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:37:25.690202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 25f73d0bd4e64057a922d11be46eb04c Jun 10 13:37:25.690202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-ServerRescueNegativeTestJSON-696024477-project Jun 10 13:37:25.690202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:37:25.690202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 6ddb4431-e631-4edd-af56-46cbe0241ca8 could not be found. Jun 10 13:37:25.690202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '6ddb4431-e631-4edd-af56-46cbe0241ca8'} Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6ddb4431-e631-4edd-af56-46cbe0241ca8 could not be found. Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:37:25.691169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6ddb4431-e631-4edd-af56-46cbe0241ca8 could not be found. Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6ddb4431-e631-4edd-af56-46cbe0241ca8 could not be found. Jun 10 13:37:25.693212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:37:25.694908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b99e8aa4-f5b7-41ea-9bdf-a24b80bef548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Returning 404 to user: Instance 6ddb4431-e631-4edd-af56-46cbe0241ca8 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:37:25.694908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b99e8aa4-f5b7-41ea-9bdf-a24b80bef548 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8" status: 404 len: 111 microversion: 2.1 time: 0.030558 Jun 10 13:37:25.694908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1487/2401] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:37:25 2022] GET /compute/v2.1/servers/6ddb4431-e631-4edd-af56-46cbe0241ca8 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:37:25.698122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-78299bbe-7b71-477c-b28e-9f12c79d6248 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:37:25.698616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-78299bbe-7b71-477c-b28e-9f12c79d6248 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:37:25.704592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78299bbe-7b71-477c-b28e-9f12c79d6248 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:25.709642 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78299bbe-7b71-477c-b28e-9f12c79d6248 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:25.709642 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78299bbe-7b71-477c-b28e-9f12c79d6248 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:25.709642 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78299bbe-7b71-477c-b28e-9f12c79d6248 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:25.731886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-78299bbe-7b71-477c-b28e-9f12c79d6248 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:37:25.999402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-78299bbe-7b71-477c-b28e-9f12c79d6248 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "DELETE /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 204 len: 0 microversion: 2.1 time: 0.304561 Jun 10 13:37:25.999995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1488/2402] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:37:25 2022] DELETE /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jun 10 13:37:26.007959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f453b67a-e95c-4f1c-8949-5091bcf992e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:37:26.008649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f453b67a-e95c-4f1c-8949-5091bcf992e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:37:26.014677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f453b67a-e95c-4f1c-8949-5091bcf992e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:26.015646 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f453b67a-e95c-4f1c-8949-5091bcf992e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:26.016393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f453b67a-e95c-4f1c-8949-5091bcf992e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:26.016853 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f453b67a-e95c-4f1c-8949-5091bcf992e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:26.143560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f453b67a-e95c-4f1c-8949-5091bcf992e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:37:26.150910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f453b67a-e95c-4f1c-8949-5091bcf992e8 tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1521 microversion: 2.1 time: 0.146870 Jun 10 13:37:26.151587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1489/2403] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:37:26 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1521 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:37:27.157440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-16df4953-5880-4742-af67-e8522f334fcc req-2fabce1c-b04b-450a-9a2b-73d6ffeae54b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5d205551-9f23-46f8-b67b-ca5bafd58182", "name": "network-vif-deleted", "tag": "9acb878a-5d99-4c1a-b520-5b9f3cdb505f"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:37:27.165970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-16df4953-5880-4742-af67-e8522f334fcc req-2fabce1c-b04b-450a-9a2b-73d6ffeae54b service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:27.166417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-16df4953-5880-4742-af67-e8522f334fcc req-2fabce1c-b04b-450a-9a2b-73d6ffeae54b service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:27.179492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-16df4953-5880-4742-af67-e8522f334fcc req-2fabce1c-b04b-450a-9a2b-73d6ffeae54b service nova] Creating event network-vif-deleted:9acb878a-5d99-4c1a-b520-5b9f3cdb505f for instance 5d205551-9f23-46f8-b67b-ca5bafd58182 on hv-cinder-maste Jun 10 13:37:27.184167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-16df4953-5880-4742-af67-e8522f334fcc req-2fabce1c-b04b-450a-9a2b-73d6ffeae54b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030436 Jun 10 13:37:27.184757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1490/2404] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:37:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:37:27.190279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-75940afc-bfc5-49fa-ae50-7694a99e945d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:37:27.190970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-75940afc-bfc5-49fa-ae50-7694a99e945d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:37:27.197250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75940afc-bfc5-49fa-ae50-7694a99e945d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:27.200971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75940afc-bfc5-49fa-ae50-7694a99e945d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:27.200971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75940afc-bfc5-49fa-ae50-7694a99e945d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:27.200971 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75940afc-bfc5-49fa-ae50-7694a99e945d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:27.263812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-75940afc-bfc5-49fa-ae50-7694a99e945d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '25f73d0bd4e64057a922d11be46eb04c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '03423e3482614374afa8c929bd931262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:37:27.272743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-75940afc-bfc5-49fa-ae50-7694a99e945d tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 200 len: 1479 microversion: 2.1 time: 0.086733 Jun 10 13:37:27.273341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1491/2405] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:37:27 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 1479 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:37:28.289578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1c9fa27f-2114-4364-9ff6-bbbc6048250f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:37:28.290225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1c9fa27f-2114-4364-9ff6-bbbc6048250f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] [instance: 5d205551-9f23-46f8-b67b-ca5bafd58182] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:37:28.296812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c9fa27f-2114-4364-9ff6-bbbc6048250f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:28.297257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c9fa27f-2114-4364-9ff6-bbbc6048250f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:28.298097 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c9fa27f-2114-4364-9ff6-bbbc6048250f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:37:28.298524 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c9fa27f-2114-4364-9ff6-bbbc6048250f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:37:28.312265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-1c9fa27f-2114-4364-9ff6-bbbc6048250f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] HTTP exception thrown: Instance 5d205551-9f23-46f8-b67b-ca5bafd58182 could not be found. Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1c9fa27f-2114-4364-9ff6-bbbc6048250f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Request method failure captured: Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 HTTP/1.1 Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0HbeUNf0EPZ-aslwQVWNd8-Z8OHNvp8Sb8m5gjvFM0tL0pvFj03ha5wm_h98nRGWdTUfHXBkNJa9XdKTFzGaRbJsRxJn5YRVRoYaqROYy-SpZpzuBlLa9e8KDT13WQC8G8gU0qsOwDmzLx6BmD9PCuaOyoARKjO1sMu6-pWpQAV11w Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 03423e3482614374afa8c929bd931262 Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-696024477 Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:37:28.313914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:37:28.315691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/03423e3482614374afa8c929bd931262"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/03423e3482614374afa8c929bd931262"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_03423e3482614374afa8c929bd931262", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/03423e3482614374afa8c929bd931262"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:37:28.315691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-ServerRescueNegativeTestJSON-696024477 Jun 10 13:37:28.315691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 03423e3482614374afa8c929bd931262 Jun 10 13:37:28.315691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-696024477 Jun 10 13:37:28.315691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-ServerRescueNegativeTestJSON-696024477-project Jun 10 13:37:28.315691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:37:28.315691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:37:28.315691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 25f73d0bd4e64057a922d11be46eb04c Jun 10 13:37:28.315691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-ServerRescueNegativeTestJSON-696024477-project Jun 10 13:37:28.315691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:37:28.315691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 5d205551-9f23-46f8-b67b-ca5bafd58182 could not be found. Jun 10 13:37:28.315691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '5d205551-9f23-46f8-b67b-ca5bafd58182'} Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5d205551-9f23-46f8-b67b-ca5bafd58182 could not be found. Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:37:28.316692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5d205551-9f23-46f8-b67b-ca5bafd58182 could not be found. Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5d205551-9f23-46f8-b67b-ca5bafd58182 could not be found. Jun 10 13:37:28.323193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:37:28.325158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1c9fa27f-2114-4364-9ff6-bbbc6048250f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] Returning 404 to user: Instance 5d205551-9f23-46f8-b67b-ca5bafd58182 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:37:28.325158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1c9fa27f-2114-4364-9ff6-bbbc6048250f tempest-ServerRescueNegativeTestJSON-696024477 tempest-ServerRescueNegativeTestJSON-696024477-project] 10.222.0.25 "GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182" status: 404 len: 111 microversion: 2.1 time: 0.029754 Jun 10 13:37:28.325158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1492/2406] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:37:28 2022] GET /compute/v2.1/servers/5d205551-9f23-46f8-b67b-ca5bafd58182 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:38:04.394026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:38:04.394663 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:38:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:39:24.064563 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=30,nova_api:UPDATE=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:39:24.065503 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=60,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:39:24.069152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2f7e7cbc-9ba2-4611-a04f-48030b3949f3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:24.069645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2f7e7cbc-9ba2-4611-a04f-48030b3949f3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:24.076283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f7e7cbc-9ba2-4611-a04f-48030b3949f3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:24.086285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f7e7cbc-9ba2-4611-a04f-48030b3949f3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:24.086285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f7e7cbc-9ba2-4611-a04f-48030b3949f3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:24.086285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2f7e7cbc-9ba2-4611-a04f-48030b3949f3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:24.104487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2f7e7cbc-9ba2-4611-a04f-48030b3949f3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:39:24.411415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2f7e7cbc-9ba2-4611-a04f-48030b3949f3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 204 len: 0 microversion: 2.1 time: 0.348620 Jun 10 13:39:24.416382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1493/2407] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:39:24 2022] DELETE /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jun 10 13:39:24.420648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f500d3cf-8ec2-46bc-93cd-94cbd3c7a868 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:24.421377 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f500d3cf-8ec2-46bc-93cd-94cbd3c7a868 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:24.427329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f500d3cf-8ec2-46bc-93cd-94cbd3c7a868 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:24.427783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f500d3cf-8ec2-46bc-93cd-94cbd3c7a868 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:24.428546 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f500d3cf-8ec2-46bc-93cd-94cbd3c7a868 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:24.429020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f500d3cf-8ec2-46bc-93cd-94cbd3c7a868 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:24.559327 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f500d3cf-8ec2-46bc-93cd-94cbd3c7a868 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:24.576245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f500d3cf-8ec2-46bc-93cd-94cbd3c7a868 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1683 microversion: 2.1 time: 0.150891 Jun 10 13:39:24.576245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1494/2408] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:24 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1683 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:25.584533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-92c1507f-76f9-4c5b-b0ca-3830c7cbd5cf tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:25.585237 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-92c1507f-76f9-4c5b-b0ca-3830c7cbd5cf tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:25.592265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-92c1507f-76f9-4c5b-b0ca-3830c7cbd5cf tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:25.592704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-92c1507f-76f9-4c5b-b0ca-3830c7cbd5cf tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:25.593485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-92c1507f-76f9-4c5b-b0ca-3830c7cbd5cf tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:25.593922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-92c1507f-76f9-4c5b-b0ca-3830c7cbd5cf tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:25.685707 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-92c1507f-76f9-4c5b-b0ca-3830c7cbd5cf tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:25.693802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-92c1507f-76f9-4c5b-b0ca-3830c7cbd5cf tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1615 microversion: 2.1 time: 0.113072 Jun 10 13:39:25.704776 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1495/2409] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:25 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1615 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:25.903826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-dc8ac1c8-22f0-4458-8712-5db548af1526 req-186f1caf-dd56-48ba-9f1d-4402f1cc0cb9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b923ce4b-4493-492c-82b8-4adc45ea47f7", "name": "network-vif-deleted", "tag": "f237b5cf-c7a4-49a8-b20d-f2f6c15cb9af"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:39:25.918585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-dc8ac1c8-22f0-4458-8712-5db548af1526 req-186f1caf-dd56-48ba-9f1d-4402f1cc0cb9 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:25.918960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-dc8ac1c8-22f0-4458-8712-5db548af1526 req-186f1caf-dd56-48ba-9f1d-4402f1cc0cb9 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:25.930010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-dc8ac1c8-22f0-4458-8712-5db548af1526 req-186f1caf-dd56-48ba-9f1d-4402f1cc0cb9 service nova] Creating event network-vif-deleted:f237b5cf-c7a4-49a8-b20d-f2f6c15cb9af for instance b923ce4b-4493-492c-82b8-4adc45ea47f7 on hv-cinder-maste Jun 10 13:39:25.934966 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-dc8ac1c8-22f0-4458-8712-5db548af1526 req-186f1caf-dd56-48ba-9f1d-4402f1cc0cb9 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035398 Jun 10 13:39:25.935556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1496/2410] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:39:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:39:26.713020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1b3e2b1d-6b92-4d52-9e70-79f84646d2c3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:26.714001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1b3e2b1d-6b92-4d52-9e70-79f84646d2c3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:26.722041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b3e2b1d-6b92-4d52-9e70-79f84646d2c3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:26.722587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b3e2b1d-6b92-4d52-9e70-79f84646d2c3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:26.723675 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b3e2b1d-6b92-4d52-9e70-79f84646d2c3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:26.724340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b3e2b1d-6b92-4d52-9e70-79f84646d2c3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:26.762456 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b3e2b1d-6b92-4d52-9e70-79f84646d2c3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:26.762905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1b3e2b1d-6b92-4d52-9e70-79f84646d2c3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:26.763413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-1b3e2b1d-6b92-4d52-9e70-79f84646d2c3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lazy-loading 'fault' on Instance uuid b923ce4b-4493-492c-82b8-4adc45ea47f7 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:39:26.810137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1b3e2b1d-6b92-4d52-9e70-79f84646d2c3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:26.818266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1b3e2b1d-6b92-4d52-9e70-79f84646d2c3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 200 len: 1332 microversion: 2.1 time: 0.110348 Jun 10 13:39:26.818830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1497/2411] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:26 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 1332 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:27.835750 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8733a006-71aa-4d2d-a571-124b12553698 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:27.836381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8733a006-71aa-4d2d-a571-124b12553698 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:27.843431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8733a006-71aa-4d2d-a571-124b12553698 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:27.843904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8733a006-71aa-4d2d-a571-124b12553698 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:27.844645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8733a006-71aa-4d2d-a571-124b12553698 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:27.845107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8733a006-71aa-4d2d-a571-124b12553698 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:27.858705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-8733a006-71aa-4d2d-a571-124b12553698 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] HTTP exception thrown: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8733a006-71aa-4d2d-a571-124b12553698 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Request method failure captured: Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 HTTP/1.1 Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0ftzakwnsjX1_z0cRv3N0JdKbd7XTWcq0bpV9CmZUkNqfVXr0KYvzMAT_PpAbli07mW_6EmovwUniFCsu9Ty1F51kbSG5WbOlXF2lGHoyN4lN1cl-t8ypmG7y3V6y5YPehLSqP2Ng8cCFZlNhxGtZaGC5bhQeMP5jiRYh304eB61I4 Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:39:27.860362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:39:27.862168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_eee8e4f43f484abdb588a8f3966d74ad", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:39:27.862168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:39:27.862168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:39:27.862168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:39:27.862168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:39:27.862168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:39:27.862168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:39:27.862168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 434ae422b68e4145ad8ecac27f450cbc Jun 10 13:39:27.862168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:39:27.862168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:39:27.862168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:39:27.862168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'b923ce4b-4493-492c-82b8-4adc45ea47f7'} Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:39:27.863158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:39:27.864926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:39:27.866457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8733a006-71aa-4d2d-a571-124b12553698 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Returning 404 to user: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:39:27.866457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8733a006-71aa-4d2d-a571-124b12553698 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 404 len: 111 microversion: 2.1 time: 0.029785 Jun 10 13:39:27.866457 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1498/2412] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:27 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:39:27.872807 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-89542800-d6e9-4c93-989a-2bcbe2561bd1 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '.version_select at 0x7f93099fe3a0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:27.983640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-89542800-d6e9-4c93-989a-2bcbe2561bd1 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1706060724" status: 202 len: 0 microversion: 2.1 time: 0.116887 Jun 10 13:39:27.984232 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1499/2413] 10.222.0.25 () {58 vars in 1220 bytes} [Fri Jun 10 13:39:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1706060724 => generated 0 bytes in 118 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:39:28.873950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0d315f15-a739-4935-b039-5cc78d981c1b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Action: 'create', calling method: .version_select at 0x7f9309954160>, body: {"keypair": {"name": "tempest-keypair-574945915", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFSN76MjCGcZctFzeZRQ3Ah659fomSfOisYKTwlMMt2QHLayfjLcUQUUlJxYvdfaMBaS9ybFLQOb9tVFhCkim0NSipIAazTEB2GhJYP0RhZooriTd+1qNOtHszOS65+haw=="}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:39:28.889381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-0d315f15-a739-4935-b039-5cc78d981c1b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Getting quotas for project eee8e4f43f484abdb588a8f3966d74ad. Resources: {'key_pairs'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:39:28.893280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-0d315f15-a739-4935-b039-5cc78d981c1b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Getting quotas for user 434ae422b68e4145ad8ecac27f450cbc and project eee8e4f43f484abdb588a8f3966d74ad. Resources: {'key_pairs'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:39:28.971904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0d315f15-a739-4935-b039-5cc78d981c1b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.102973 Jun 10 13:39:28.972495 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1500/2414] 10.222.0.25 () {60 vars in 1158 bytes} [Fri Jun 10 13:39:28 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:39:30.996720 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1437878084", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e47293bb-c3a4-4f5c-acfc-80db56fb3d26"}], "security_groups": [{"name": "tempest-securitygroup--1009075146"}], "key_name": "tempest-keypair-574945915", "user_data": "}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:39:31.095584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.network.neutron [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] validate_networks() for [('e47293bb-c3a4-4f5c-acfc-80db56fb3d26', None, None, None, None, None)] {{(pid=118092) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:39:31.558451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Flavor limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:39:31.558871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Image limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:39:31.559364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Flavor pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:39:31.559766 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Image pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:39:31.560386 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:39:31.578184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:31.578628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:31.579418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:31.579859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:31.596709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Getting quotas for project eee8e4f43f484abdb588a8f3966d74ad. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:39:31.600648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Getting quotas for user 434ae422b68e4145ad8ecac27f450cbc and project eee8e4f43f484abdb588a8f3966d74ad. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:39:31.612143 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Going to run 1 instances... {{(pid=118092) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:39:31.614432 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118092) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:39:31.727787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7514b0e7-bffe-4235-a751-5ce374f76375 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.735642 Jun 10 13:39:31.728574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1501/2415] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:39:30 2022] POST /compute/v2.1/servers => generated 402 bytes in 737 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jun 10 13:39:31.737979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4a509fb0-6b2b-435a-8b86-70025b869f0a tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:31.738583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4a509fb0-6b2b-435a-8b86-70025b869f0a tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:31.790612 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4a509fb0-6b2b-435a-8b86-70025b869f0a tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:31.797370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4a509fb0-6b2b-435a-8b86-70025b869f0a tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1241 microversion: 2.1 time: 0.063002 Jun 10 13:39:31.798139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1502/2416] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:31 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1241 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:32.814626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d184ecd1-18f8-43d5-a579-8e8529757b5f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:32.815331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d184ecd1-18f8-43d5-a579-8e8529757b5f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:32.822378 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d184ecd1-18f8-43d5-a579-8e8529757b5f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:32.822825 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d184ecd1-18f8-43d5-a579-8e8529757b5f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:32.823581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d184ecd1-18f8-43d5-a579-8e8529757b5f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:32.824015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d184ecd1-18f8-43d5-a579-8e8529757b5f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:32.883683 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d184ecd1-18f8-43d5-a579-8e8529757b5f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:32.891219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d184ecd1-18f8-43d5-a579-8e8529757b5f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1245 microversion: 2.1 time: 0.080872 Jun 10 13:39:32.891784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1503/2417] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:32 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1245 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:33.907602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9b3d3d04-79cd-4899-86d4-9e31c0394116 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:33.908345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9b3d3d04-79cd-4899-86d4-9e31c0394116 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:33.914785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9b3d3d04-79cd-4899-86d4-9e31c0394116 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:33.915239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9b3d3d04-79cd-4899-86d4-9e31c0394116 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:33.916031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9b3d3d04-79cd-4899-86d4-9e31c0394116 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:33.916464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9b3d3d04-79cd-4899-86d4-9e31c0394116 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:33.979647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9b3d3d04-79cd-4899-86d4-9e31c0394116 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:33.987717 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9b3d3d04-79cd-4899-86d4-9e31c0394116 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1293 microversion: 2.1 time: 0.084116 Jun 10 13:39:33.988299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1504/2418] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:33 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1293 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:35.005711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bc1acf44-13a6-43a1-a789-75d9867cc722 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:35.006315 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bc1acf44-13a6-43a1-a789-75d9867cc722 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:35.013264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bc1acf44-13a6-43a1-a789-75d9867cc722 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:35.013712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bc1acf44-13a6-43a1-a789-75d9867cc722 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:35.014788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bc1acf44-13a6-43a1-a789-75d9867cc722 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:35.015092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bc1acf44-13a6-43a1-a789-75d9867cc722 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:35.180017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bc1acf44-13a6-43a1-a789-75d9867cc722 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:35.187995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bc1acf44-13a6-43a1-a789-75d9867cc722 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1367 microversion: 2.1 time: 0.186139 Jun 10 13:39:35.188847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1505/2419] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:34 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1367 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:39:35.918372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-5f09ea30-6c1e-4473-ae9c-abf4b761b9dc req-f616de9c-179d-4d93-a3dd-49fb1e2ffc21 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fca751de-4ecc-444e-af32-ea1b33e262e7", "tag": "d82a0cc9-052d-4e81-8ad7-be11cef48d54"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:39:35.929431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-5f09ea30-6c1e-4473-ae9c-abf4b761b9dc req-f616de9c-179d-4d93-a3dd-49fb1e2ffc21 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:35.929882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-5f09ea30-6c1e-4473-ae9c-abf4b761b9dc req-f616de9c-179d-4d93-a3dd-49fb1e2ffc21 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:35.944626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-5f09ea30-6c1e-4473-ae9c-abf4b761b9dc req-f616de9c-179d-4d93-a3dd-49fb1e2ffc21 service nova] Creating event network-changed:d82a0cc9-052d-4e81-8ad7-be11cef48d54 for instance fca751de-4ecc-444e-af32-ea1b33e262e7 on hv-cinder-maste Jun 10 13:39:35.950282 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-5f09ea30-6c1e-4473-ae9c-abf4b761b9dc req-f616de9c-179d-4d93-a3dd-49fb1e2ffc21 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035625 Jun 10 13:39:35.950884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1506/2420] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:39:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:39:36.207448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0f5f2d61-548b-48ba-ba1f-057cf0fbf382 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:36.208714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0f5f2d61-548b-48ba-ba1f-057cf0fbf382 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:36.217219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f5f2d61-548b-48ba-ba1f-057cf0fbf382 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:36.217674 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f5f2d61-548b-48ba-ba1f-057cf0fbf382 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:36.218485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f5f2d61-548b-48ba-ba1f-057cf0fbf382 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:36.218949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0f5f2d61-548b-48ba-ba1f-057cf0fbf382 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:36.358702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0f5f2d61-548b-48ba-ba1f-057cf0fbf382 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:36.366841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0f5f2d61-548b-48ba-ba1f-057cf0fbf382 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1367 microversion: 2.1 time: 0.164590 Jun 10 13:39:36.367408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1507/2421] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:36 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1367 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:37.383911 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-691e45f4-7fe2-4a36-97db-5d71d9057d92 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:37.384520 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-691e45f4-7fe2-4a36-97db-5d71d9057d92 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:37.391990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-691e45f4-7fe2-4a36-97db-5d71d9057d92 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:37.392437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-691e45f4-7fe2-4a36-97db-5d71d9057d92 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:37.393217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-691e45f4-7fe2-4a36-97db-5d71d9057d92 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:37.393649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-691e45f4-7fe2-4a36-97db-5d71d9057d92 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:37.535776 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-691e45f4-7fe2-4a36-97db-5d71d9057d92 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:37.543274 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-691e45f4-7fe2-4a36-97db-5d71d9057d92 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1367 microversion: 2.1 time: 0.163189 Jun 10 13:39:37.543827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1508/2422] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:37 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1367 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:38.560463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a620e0ea-6d20-4b40-8273-db53e4cd6f0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:38.561115 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a620e0ea-6d20-4b40-8273-db53e4cd6f0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:38.567821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a620e0ea-6d20-4b40-8273-db53e4cd6f0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:38.568265 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a620e0ea-6d20-4b40-8273-db53e4cd6f0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:38.569080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a620e0ea-6d20-4b40-8273-db53e4cd6f0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:38.569511 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a620e0ea-6d20-4b40-8273-db53e4cd6f0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:38.692056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a620e0ea-6d20-4b40-8273-db53e4cd6f0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:38.699285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a620e0ea-6d20-4b40-8273-db53e4cd6f0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1367 microversion: 2.1 time: 0.142812 Jun 10 13:39:38.699838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1509/2423] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:38 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1367 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:39.716470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dc400d05-1be9-45e3-bb17-057c92066bb3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:39.717137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dc400d05-1be9-45e3-bb17-057c92066bb3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:39.724401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc400d05-1be9-45e3-bb17-057c92066bb3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:39.724876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc400d05-1be9-45e3-bb17-057c92066bb3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:39.725622 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc400d05-1be9-45e3-bb17-057c92066bb3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:39.726050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dc400d05-1be9-45e3-bb17-057c92066bb3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:39.856962 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dc400d05-1be9-45e3-bb17-057c92066bb3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:39.864511 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dc400d05-1be9-45e3-bb17-057c92066bb3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1367 microversion: 2.1 time: 0.151868 Jun 10 13:39:39.865096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1510/2424] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:39 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1367 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:40.881666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-466a8c3e-f7d3-4a88-815b-62ec37d3b684 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:40.882259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-466a8c3e-f7d3-4a88-815b-62ec37d3b684 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:40.888666 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-466a8c3e-f7d3-4a88-815b-62ec37d3b684 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:40.889156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-466a8c3e-f7d3-4a88-815b-62ec37d3b684 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:40.889914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-466a8c3e-f7d3-4a88-815b-62ec37d3b684 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:40.890340 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-466a8c3e-f7d3-4a88-815b-62ec37d3b684 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:41.017016 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-466a8c3e-f7d3-4a88-815b-62ec37d3b684 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:41.024413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-466a8c3e-f7d3-4a88-815b-62ec37d3b684 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1367 microversion: 2.1 time: 0.146546 Jun 10 13:39:41.025019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1511/2425] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:40 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1367 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:42.041765 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-43527237-1b1b-4d88-b5a8-4267c1c718dc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:42.042373 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-43527237-1b1b-4d88-b5a8-4267c1c718dc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:42.044347 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:39:42.045064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=12 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:39:42.050275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-43527237-1b1b-4d88-b5a8-4267c1c718dc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:42.050730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-43527237-1b1b-4d88-b5a8-4267c1c718dc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:42.051490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-43527237-1b1b-4d88-b5a8-4267c1c718dc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:42.051922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-43527237-1b1b-4d88-b5a8-4267c1c718dc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:42.176474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-43527237-1b1b-4d88-b5a8-4267c1c718dc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:42.183701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-43527237-1b1b-4d88-b5a8-4267c1c718dc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1367 microversion: 2.1 time: 0.145810 Jun 10 13:39:42.184259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1512/2426] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:42 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1367 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:42.614969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-c8c3336e-ca47-4e08-9658-108a4394a024 req-c27dd53c-ba39-4fec-8a2d-faa67866d837 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fca751de-4ecc-444e-af32-ea1b33e262e7", "name": "network-vif-plugged", "status": "completed", "tag": "d82a0cc9-052d-4e81-8ad7-be11cef48d54"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:39:42.623071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-c8c3336e-ca47-4e08-9658-108a4394a024 req-c27dd53c-ba39-4fec-8a2d-faa67866d837 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:42.628525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-c8c3336e-ca47-4e08-9658-108a4394a024 req-c27dd53c-ba39-4fec-8a2d-faa67866d837 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:42.628525 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:39:42.634649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-c8c3336e-ca47-4e08-9658-108a4394a024 req-c27dd53c-ba39-4fec-8a2d-faa67866d837 service nova] Creating event network-vif-plugged:d82a0cc9-052d-4e81-8ad7-be11cef48d54 for instance fca751de-4ecc-444e-af32-ea1b33e262e7 on hv-cinder-maste Jun 10 13:39:42.639604 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-c8c3336e-ca47-4e08-9658-108a4394a024 req-c27dd53c-ba39-4fec-8a2d-faa67866d837 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.166513 Jun 10 13:39:42.640173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1513/2427] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:39:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:39:43.202504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7c99a427-ceb5-4ccd-84f6-ebd6289c028b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:43.203151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7c99a427-ceb5-4ccd-84f6-ebd6289c028b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:43.209951 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7c99a427-ceb5-4ccd-84f6-ebd6289c028b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:43.210400 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7c99a427-ceb5-4ccd-84f6-ebd6289c028b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:43.211153 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7c99a427-ceb5-4ccd-84f6-ebd6289c028b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:43.211589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7c99a427-ceb5-4ccd-84f6-ebd6289c028b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:43.342733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7c99a427-ceb5-4ccd-84f6-ebd6289c028b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:43.350774 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7c99a427-ceb5-4ccd-84f6-ebd6289c028b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1367 microversion: 2.1 time: 0.153115 Jun 10 13:39:43.351357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1514/2428] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:43 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1367 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:44.367420 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a8f7b579-e582-4e36-acaa-63448b27fc0c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:44.368023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a8f7b579-e582-4e36-acaa-63448b27fc0c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:44.375227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8f7b579-e582-4e36-acaa-63448b27fc0c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:44.375673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8f7b579-e582-4e36-acaa-63448b27fc0c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:44.376420 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8f7b579-e582-4e36-acaa-63448b27fc0c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:44.376875 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a8f7b579-e582-4e36-acaa-63448b27fc0c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:44.503313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a8f7b579-e582-4e36-acaa-63448b27fc0c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:44.507687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a8f7b579-e582-4e36-acaa-63448b27fc0c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1367 microversion: 2.1 time: 0.144102 Jun 10 13:39:44.508249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1515/2429] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:44 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1367 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:45.524554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-860d0e98-c1e6-47f2-a2c8-a0ab22ad0a84 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:45.525246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-860d0e98-c1e6-47f2-a2c8-a0ab22ad0a84 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:45.532273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-860d0e98-c1e6-47f2-a2c8-a0ab22ad0a84 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:45.532716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-860d0e98-c1e6-47f2-a2c8-a0ab22ad0a84 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:45.533488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-860d0e98-c1e6-47f2-a2c8-a0ab22ad0a84 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:45.536838 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-860d0e98-c1e6-47f2-a2c8-a0ab22ad0a84 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:45.671181 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-860d0e98-c1e6-47f2-a2c8-a0ab22ad0a84 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:39:45.679192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-860d0e98-c1e6-47f2-a2c8-a0ab22ad0a84 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1557 microversion: 2.1 time: 0.158974 Jun 10 13:39:45.679752 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1516/2430] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:39:45 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1557 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:39:45.694914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e1833713-a7e9-4a83-990a-5c232ab7e94b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:39:45.695406 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e1833713-a7e9-4a83-990a-5c232ab7e94b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:39:45.700948 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e1833713-a7e9-4a83-990a-5c232ab7e94b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:45.701392 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e1833713-a7e9-4a83-990a-5c232ab7e94b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:45.702133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e1833713-a7e9-4a83-990a-5c232ab7e94b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:45.702560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e1833713-a7e9-4a83-990a-5c232ab7e94b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:45.775266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e1833713-a7e9-4a83-990a-5c232ab7e94b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.083593 Jun 10 13:39:45.775856 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1517/2431] 10.222.0.25 () {58 vars in 1274 bytes} [Fri Jun 10 13:39:45 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-interface => generated 285 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:39:46.950209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-f247719c-6f29-4258-a986-d1f29362cdb1 req-fb72b311-2cfd-486c-a5ae-a77d0c967d51 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fca751de-4ecc-444e-af32-ea1b33e262e7", "tag": "d82a0cc9-052d-4e81-8ad7-be11cef48d54"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:39:46.958602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-f247719c-6f29-4258-a986-d1f29362cdb1 req-fb72b311-2cfd-486c-a5ae-a77d0c967d51 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:39:46.959040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-f247719c-6f29-4258-a986-d1f29362cdb1 req-fb72b311-2cfd-486c-a5ae-a77d0c967d51 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:39:46.971326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-f247719c-6f29-4258-a986-d1f29362cdb1 req-fb72b311-2cfd-486c-a5ae-a77d0c967d51 service nova] Creating event network-changed:d82a0cc9-052d-4e81-8ad7-be11cef48d54 for instance fca751de-4ecc-444e-af32-ea1b33e262e7 on hv-cinder-maste Jun 10 13:39:46.976578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-f247719c-6f29-4258-a986-d1f29362cdb1 req-fb72b311-2cfd-486c-a5ae-a77d0c967d51 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.085928 Jun 10 13:39:46.977380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1518/2432] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:39:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:40:22.914903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7ff56669-39e4-4813-90ff-3a944255e4e9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:22.915813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7ff56669-39e4-4813-90ff-3a944255e4e9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:22.918106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=84,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:40:22.918890 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=24 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:40:22.924515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ff56669-39e4-4813-90ff-3a944255e4e9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:22.925101 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ff56669-39e4-4813-90ff-3a944255e4e9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:22.934128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ff56669-39e4-4813-90ff-3a944255e4e9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:22.934128 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ff56669-39e4-4813-90ff-3a944255e4e9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:22.953802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7ff56669-39e4-4813-90ff-3a944255e4e9 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/ips" status: 200 len: 150 microversion: 2.1 time: 0.043280 Jun 10 13:40:22.959465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1519/2433] 10.222.0.25 () {58 vars in 1247 bytes} [Fri Jun 10 13:40:22 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/ips => generated 150 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:40:24.574780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e34986c6-1ce3-46aa-82a8-ac198c05f490 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3c7af3af-e29f-4bf7-8b71-1f74a509bb1b"}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:40:24.577658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e34986c6-1ce3-46aa-82a8-ac198c05f490 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:24.584082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e34986c6-1ce3-46aa-82a8-ac198c05f490 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:24.584522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e34986c6-1ce3-46aa-82a8-ac198c05f490 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:24.585493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e34986c6-1ce3-46aa-82a8-ac198c05f490 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:24.585924 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e34986c6-1ce3-46aa-82a8-ac198c05f490 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:25.418414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e34986c6-1ce3-46aa-82a8-ac198c05f490 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "POST /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.848122 Jun 10 13:40:25.419038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1520/2434] 10.222.0.25 () {60 vars in 1322 bytes} [Fri Jun 10 13:40:24 2022] POST /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-volume_attachments => generated 194 bytes in 850 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:40:34.001440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e4d05a87-d9d7-4b68-91b2-8c7d0e47cac8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3c7af3af-e29f-4bf7-8b71-1f74a509bb1b"}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:40:34.003332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e4d05a87-d9d7-4b68-91b2-8c7d0e47cac8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:34.009541 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4d05a87-d9d7-4b68-91b2-8c7d0e47cac8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:34.010054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4d05a87-d9d7-4b68-91b2-8c7d0e47cac8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:34.010896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4d05a87-d9d7-4b68-91b2-8c7d0e47cac8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:34.011401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e4d05a87-d9d7-4b68-91b2-8c7d0e47cac8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:34.094595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-e4d05a87-d9d7-4b68-91b2-8c7d0e47cac8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] HTTP exception thrown: Invalid volume: volume 3c7af3af-e29f-4bf7-8b71-1f74a509bb1b is already attached to instances: fca751de-4ecc-444e-af32-ea1b33e262e7 Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e4d05a87-d9d7-4b68-91b2-8c7d0e47cac8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Request method failure captured: Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: POST /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-volume_attachments HTTP/1.1 Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 74 Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0ftzakwnsjX1_z0cRv3N0JdKbd7XTWcq0bpV9CmZUkNqfVXr0KYvzMAT_PpAbli07mW_6EmovwUniFCsu9Ty1F51kbSG5WbOlXF2lGHoyN4lN1cl-t8ypmG7y3V6y5YPehLSqP2Ng8cCFZlNhxGtZaGC5bhQeMP5jiRYh304eB61I4 Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:40:34.096647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:40:34.098648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_eee8e4f43f484abdb588a8f3966d74ad", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:40:34.098648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:40:34.098648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:40:34.098648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:40:34.098648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:40:34.098648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:40:34.098648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:40:34.098648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 434ae422b68e4145ad8ecac27f450cbc Jun 10 13:40:34.098648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:40:34.098648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Jun 10 13:40:34.098648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {"volumeAttachment": {"volumeId": "3c7af3af-e29f-4bf7-8b71-1f74a509bb1b"}} Jun 10 13:40:34.098648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Invalid volume: volume 3c7af3af-e29f-4bf7-8b71-1f74a509bb1b is already attached to instances: fca751de-4ecc-444e-af32-ea1b33e262e7 Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'server_id': 'fca751de-4ecc-444e-af32-ea1b33e262e7', 'body': {'volumeAttachment': {'volumeId': '3c7af3af-e29f-4bf7-8b71-1f74a509bb1b'}}} Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 3c7af3af-e29f-4bf7-8b71-1f74a509bb1b is already attached to instances: fca751de-4ecc-444e-af32-ea1b33e262e7 Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:40:34.099657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 3c7af3af-e29f-4bf7-8b71-1f74a509bb1b is already attached to instances: fca751de-4ecc-444e-af32-ea1b33e262e7 Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:40:34.102786 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:40:34.104969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:40:34.104969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:40:34.104969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:40:34.104969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 3c7af3af-e29f-4bf7-8b71-1f74a509bb1b is already attached to instances: fca751de-4ecc-444e-af32-ea1b33e262e7 Jun 10 13:40:34.104969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:40:34.104969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e4d05a87-d9d7-4b68-91b2-8c7d0e47cac8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Returning 400 to user: Invalid volume: volume 3c7af3af-e29f-4bf7-8b71-1f74a509bb1b is already attached to instances: fca751de-4ecc-444e-af32-ea1b33e262e7 {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:40:34.104969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e4d05a87-d9d7-4b68-91b2-8c7d0e47cac8 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "POST /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.102544 Jun 10 13:40:34.104969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1521/2435] 10.222.0.25 () {60 vars in 1322 bytes} [Fri Jun 10 13:40:33 2022] POST /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-volume_attachments => generated 174 bytes in 104 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jun 10 13:40:34.159781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e59810e5-3105-4fd5-97b5-305b3016b155 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:34.160225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e59810e5-3105-4fd5-97b5-305b3016b155 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:34.166240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e59810e5-3105-4fd5-97b5-305b3016b155 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:34.166709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e59810e5-3105-4fd5-97b5-305b3016b155 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:34.167448 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e59810e5-3105-4fd5-97b5-305b3016b155 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:34.167881 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e59810e5-3105-4fd5-97b5-305b3016b155 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:34.252835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-e59810e5-3105-4fd5-97b5-305b3016b155 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lazy-loading 'services' on Instance uuid fca751de-4ecc-444e-af32-ea1b33e262e7 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:40:34.443381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e59810e5-3105-4fd5-97b5-305b3016b155 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-volume_attachments/3c7af3af-e29f-4bf7-8b71-1f74a509bb1b" status: 202 len: 0 microversion: 2.1 time: 0.287403 Jun 10 13:40:34.443994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1522/2436] 10.222.0.25 () {58 vars in 1415 bytes} [Fri Jun 10 13:40:34 2022] DELETE /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-volume_attachments/3c7af3af-e29f-4bf7-8b71-1f74a509bb1b => generated 0 bytes in 289 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:40:34.456399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-de7230f9-cdfb-4f14-8d13-a99154344b6a tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:34.457679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-de7230f9-cdfb-4f14-8d13-a99154344b6a tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:34.465235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-de7230f9-cdfb-4f14-8d13-a99154344b6a tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:34.465753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-de7230f9-cdfb-4f14-8d13-a99154344b6a tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:34.467001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-de7230f9-cdfb-4f14-8d13-a99154344b6a tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:34.468001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-de7230f9-cdfb-4f14-8d13-a99154344b6a tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:34.497773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-de7230f9-cdfb-4f14-8d13-a99154344b6a tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049078 Jun 10 13:40:34.498291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1523/2437] 10.222.0.25 () {58 vars in 1301 bytes} [Fri Jun 10 13:40:34 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:40:35.508883 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6eb55c5e-9253-4456-b1c7-d1321c4c668d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:35.509761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6eb55c5e-9253-4456-b1c7-d1321c4c668d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:35.516180 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6eb55c5e-9253-4456-b1c7-d1321c4c668d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:35.516607 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6eb55c5e-9253-4456-b1c7-d1321c4c668d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:35.517447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6eb55c5e-9253-4456-b1c7-d1321c4c668d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:35.518383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6eb55c5e-9253-4456-b1c7-d1321c4c668d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:35.547626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6eb55c5e-9253-4456-b1c7-d1321c4c668d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042871 Jun 10 13:40:35.548191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1524/2438] 10.222.0.25 () {58 vars in 1301 bytes} [Fri Jun 10 13:40:35 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:40:36.558849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9edc5e9e-cff3-4d1a-810b-2b11c5642093 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:36.559749 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9edc5e9e-cff3-4d1a-810b-2b11c5642093 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:36.567636 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9edc5e9e-cff3-4d1a-810b-2b11c5642093 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:36.568096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9edc5e9e-cff3-4d1a-810b-2b11c5642093 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:36.568892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9edc5e9e-cff3-4d1a-810b-2b11c5642093 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:36.569429 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9edc5e9e-cff3-4d1a-810b-2b11c5642093 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:36.603131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9edc5e9e-cff3-4d1a-810b-2b11c5642093 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048193 Jun 10 13:40:36.603753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1525/2439] 10.222.0.25 () {58 vars in 1301 bytes} [Fri Jun 10 13:40:36 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jun 10 13:40:36.660925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-39ecfa4d-e65c-459b-8b0e-68290c10164e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:36.661451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-39ecfa4d-e65c-459b-8b0e-68290c10164e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:36.667948 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39ecfa4d-e65c-459b-8b0e-68290c10164e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:36.667948 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39ecfa4d-e65c-459b-8b0e-68290c10164e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:36.668644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39ecfa4d-e65c-459b-8b0e-68290c10164e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:36.669240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39ecfa4d-e65c-459b-8b0e-68290c10164e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:36.695212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-39ecfa4d-e65c-459b-8b0e-68290c10164e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:40:37.054526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-39ecfa4d-e65c-459b-8b0e-68290c10164e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 204 len: 0 microversion: 2.1 time: 0.397677 Jun 10 13:40:37.055431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1526/2440] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:40:36 2022] DELETE /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jun 10 13:40:37.066515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-510f71fb-0595-463c-b61b-edca1374d638 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:37.066515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-510f71fb-0595-463c-b61b-edca1374d638 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:37.071804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-510f71fb-0595-463c-b61b-edca1374d638 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:37.072251 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-510f71fb-0595-463c-b61b-edca1374d638 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:37.073050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-510f71fb-0595-463c-b61b-edca1374d638 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:37.073480 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-510f71fb-0595-463c-b61b-edca1374d638 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:37.240722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-510f71fb-0595-463c-b61b-edca1374d638 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:37.248657 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-510f71fb-0595-463c-b61b-edca1374d638 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1682 microversion: 2.1 time: 0.188942 Jun 10 13:40:37.249245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1527/2441] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:37 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1682 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:40:38.265958 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4bfbf91a-09ff-4a84-8dda-7a99d3c315ad tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:38.266641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4bfbf91a-09ff-4a84-8dda-7a99d3c315ad tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:38.273823 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4bfbf91a-09ff-4a84-8dda-7a99d3c315ad tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:38.274213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4bfbf91a-09ff-4a84-8dda-7a99d3c315ad tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:38.274978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4bfbf91a-09ff-4a84-8dda-7a99d3c315ad tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:38.275417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4bfbf91a-09ff-4a84-8dda-7a99d3c315ad tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:38.444322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4bfbf91a-09ff-4a84-8dda-7a99d3c315ad tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:38.455403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4bfbf91a-09ff-4a84-8dda-7a99d3c315ad tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1682 microversion: 2.1 time: 0.193214 Jun 10 13:40:38.457422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1528/2442] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:38 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1682 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:40:39.476139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-91eee887-dd33-4a2f-823e-07217497866c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:39.476798 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-91eee887-dd33-4a2f-823e-07217497866c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:39.483860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91eee887-dd33-4a2f-823e-07217497866c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:39.484404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91eee887-dd33-4a2f-823e-07217497866c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:39.491644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91eee887-dd33-4a2f-823e-07217497866c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:39.491644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91eee887-dd33-4a2f-823e-07217497866c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:39.740982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-91eee887-dd33-4a2f-823e-07217497866c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:39.749222 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-91eee887-dd33-4a2f-823e-07217497866c tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1682 microversion: 2.1 time: 0.276905 Jun 10 13:40:39.750178 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1529/2443] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:39 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1682 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:40:39.996442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-6a016c6b-39fb-4c59-b600-a74fe88002ac req-29c75d23-66f0-4bec-aaf9-7484fe9557df service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fca751de-4ecc-444e-af32-ea1b33e262e7", "name": "network-vif-deleted", "tag": "d82a0cc9-052d-4e81-8ad7-be11cef48d54"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:40:40.007978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-6a016c6b-39fb-4c59-b600-a74fe88002ac req-29c75d23-66f0-4bec-aaf9-7484fe9557df service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:40.009320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-6a016c6b-39fb-4c59-b600-a74fe88002ac req-29c75d23-66f0-4bec-aaf9-7484fe9557df service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:40.229880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-6a016c6b-39fb-4c59-b600-a74fe88002ac req-29c75d23-66f0-4bec-aaf9-7484fe9557df service nova] Creating event network-vif-deleted:d82a0cc9-052d-4e81-8ad7-be11cef48d54 for instance fca751de-4ecc-444e-af32-ea1b33e262e7 on hv-cinder-maste Jun 10 13:40:40.235127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-6a016c6b-39fb-4c59-b600-a74fe88002ac req-29c75d23-66f0-4bec-aaf9-7484fe9557df service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.243069 Jun 10 13:40:40.247409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1530/2444] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:40:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:40:40.768730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6d2bac11-bd1b-4f05-a121-db56c5b59639 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:40.769487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6d2bac11-bd1b-4f05-a121-db56c5b59639 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:40.777550 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d2bac11-bd1b-4f05-a121-db56c5b59639 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:40.778314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d2bac11-bd1b-4f05-a121-db56c5b59639 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:40.779312 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d2bac11-bd1b-4f05-a121-db56c5b59639 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:40.779956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d2bac11-bd1b-4f05-a121-db56c5b59639 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:40.825333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d2bac11-bd1b-4f05-a121-db56c5b59639 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:40.825983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6d2bac11-bd1b-4f05-a121-db56c5b59639 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:40.826651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-6d2bac11-bd1b-4f05-a121-db56c5b59639 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lazy-loading 'fault' on Instance uuid fca751de-4ecc-444e-af32-ea1b33e262e7 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:40:41.189267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6d2bac11-bd1b-4f05-a121-db56c5b59639 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:41.197808 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6d2bac11-bd1b-4f05-a121-db56c5b59639 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 200 len: 1331 microversion: 2.1 time: 0.435396 Jun 10 13:40:41.198461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1531/2445] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:40 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 1331 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:40:42.215943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-acb18189-5494-4a72-a5e9-70bde10d3c69 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:42.216567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-acb18189-5494-4a72-a5e9-70bde10d3c69 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:42.224011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acb18189-5494-4a72-a5e9-70bde10d3c69 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:42.224744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acb18189-5494-4a72-a5e9-70bde10d3c69 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:42.225535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acb18189-5494-4a72-a5e9-70bde10d3c69 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:42.226200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acb18189-5494-4a72-a5e9-70bde10d3c69 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:42.240462 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-acb18189-5494-4a72-a5e9-70bde10d3c69 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] HTTP exception thrown: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-acb18189-5494-4a72-a5e9-70bde10d3c69 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Request method failure captured: Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 HTTP/1.1 Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0ftzakwnsjX1_z0cRv3N0JdKbd7XTWcq0bpV9CmZUkNqfVXr0KYvzMAT_PpAbli07mW_6EmovwUniFCsu9Ty1F51kbSG5WbOlXF2lGHoyN4lN1cl-t8ypmG7y3V6y5YPehLSqP2Ng8cCFZlNhxGtZaGC5bhQeMP5jiRYh304eB61I4 Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:40:42.242194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:40:42.244012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_eee8e4f43f484abdb588a8f3966d74ad", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:40:42.244012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:40:42.244012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:40:42.244012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:40:42.244012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:40:42.244012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:40:42.244012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:40:42.244012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 434ae422b68e4145ad8ecac27f450cbc Jun 10 13:40:42.244012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:40:42.244012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:40:42.244012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:40:42.244012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'fca751de-4ecc-444e-af32-ea1b33e262e7'} Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:40:42.245082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:40:42.246758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:40:42.249009 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-acb18189-5494-4a72-a5e9-70bde10d3c69 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Returning 404 to user: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:40:42.250036 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-acb18189-5494-4a72-a5e9-70bde10d3c69 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 404 len: 111 microversion: 2.1 time: 0.038522 Jun 10 13:40:42.250587 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1532/2446] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:42 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:40:42.259202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-efe2a6bc-eee4-45d6-a55f-0c09591b4e72 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '.version_select at 0x7f930993a4c0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:42.341226 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-efe2a6bc-eee4-45d6-a55f-0c09591b4e72 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-574945915" status: 202 len: 0 microversion: 2.1 time: 0.085584 Jun 10 13:40:42.343057 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1533/2447] 10.222.0.25 () {58 vars in 1217 bytes} [Fri Jun 10 13:40:42 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-574945915 => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:40:43.323065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2eab8d0e-4689-4127-b4f6-0a1b13b2189e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Action: 'create', calling method: .version_select at 0x7f93098f0af0>, body: {"keypair": {"name": "tempest-keypair-746089609", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCUPKyPNZg6Qv6MjTl7cnYQphSSMJvuhbXX5ioy34VE2xuk7bTeFr76yDibJWlqWivA9vAheN0mn9BnUbLDvyJqKvaih/cewh2k6lEg44OxSbM4Urxmi2iEr96wkXsRw+A=="}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:40:43.346114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-2eab8d0e-4689-4127-b4f6-0a1b13b2189e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Getting quotas for project eee8e4f43f484abdb588a8f3966d74ad. Resources: {'key_pairs'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:40:43.349982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-2eab8d0e-4689-4127-b4f6-0a1b13b2189e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Getting quotas for user 434ae422b68e4145ad8ecac27f450cbc and project eee8e4f43f484abdb588a8f3966d74ad. Resources: {'key_pairs'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:40:43.433316 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2eab8d0e-4689-4127-b4f6-0a1b13b2189e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.116342 Jun 10 13:40:43.436861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1534/2448] 10.222.0.25 () {60 vars in 1158 bytes} [Fri Jun 10 13:40:43 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:40:46.247051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-828820377", "imageRef": "aaa7ddf1-dd90-430c-9270-bb8f07ec162e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e47293bb-c3a4-4f5c-acfc-80db56fb3d26"}], "security_groups": [{"name": "tempest-securitygroup--479856687"}], "key_name": "tempest-keypair-746089609", "user_data": "}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:40:46.366980 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.network.neutron [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] validate_networks() for [('e47293bb-c3a4-4f5c-acfc-80db56fb3d26', None, None, None, None, None)] {{(pid=118092) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:40:47.335230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Flavor limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:40:47.335852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Image limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:40:47.337130 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Flavor pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:40:47.337959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Image pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:40:47.338430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:40:47.364932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:47.368214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:47.369266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:47.370038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:47.389636 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:40:47.400576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Getting quotas for project eee8e4f43f484abdb588a8f3966d74ad. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:40:47.410088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Getting quotas for user 434ae422b68e4145ad8ecac27f450cbc and project eee8e4f43f484abdb588a8f3966d74ad. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:40:47.431424 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Going to run 1 instances... {{(pid=118092) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:40:47.434060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='aaa7ddf1-dd90-430c-9270-bb8f07ec162e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118092) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:40:47.574627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-948a7d55-0063-496d-86f8-2122d3b26af2 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.324813 Jun 10 13:40:47.575424 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1535/2449] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:40:46 2022] POST /compute/v2.1/servers => generated 401 bytes in 1334 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jun 10 13:40:47.587764 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4335e14e-2632-41a4-965d-ec1b4292f300 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:47.592325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4335e14e-2632-41a4-965d-ec1b4292f300 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:47.662272 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-4335e14e-2632-41a4-965d-ec1b4292f300 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:47.671812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4335e14e-2632-41a4-965d-ec1b4292f300 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1240 microversion: 2.1 time: 0.088121 Jun 10 13:40:47.673990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1536/2450] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:47 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1240 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:40:48.691260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-693695b6-cbcb-46cc-9d63-a6cc2cc4cf5b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:48.691926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-693695b6-cbcb-46cc-9d63-a6cc2cc4cf5b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:48.699320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-693695b6-cbcb-46cc-9d63-a6cc2cc4cf5b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:48.699762 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-693695b6-cbcb-46cc-9d63-a6cc2cc4cf5b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:48.700516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-693695b6-cbcb-46cc-9d63-a6cc2cc4cf5b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:48.700972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-693695b6-cbcb-46cc-9d63-a6cc2cc4cf5b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:48.775625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-693695b6-cbcb-46cc-9d63-a6cc2cc4cf5b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:48.785118 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-693695b6-cbcb-46cc-9d63-a6cc2cc4cf5b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1244 microversion: 2.1 time: 0.098210 Jun 10 13:40:48.785729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1537/2451] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:48 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1244 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:40:49.812083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c71cbc68-87ff-4a44-8514-c42a9570d424 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:49.815357 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c71cbc68-87ff-4a44-8514-c42a9570d424 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:49.831455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c71cbc68-87ff-4a44-8514-c42a9570d424 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:49.832423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c71cbc68-87ff-4a44-8514-c42a9570d424 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:49.834996 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c71cbc68-87ff-4a44-8514-c42a9570d424 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:49.836545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c71cbc68-87ff-4a44-8514-c42a9570d424 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:49.943956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c71cbc68-87ff-4a44-8514-c42a9570d424 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:49.952668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c71cbc68-87ff-4a44-8514-c42a9570d424 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1292 microversion: 2.1 time: 0.152616 Jun 10 13:40:49.957945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1538/2452] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:49 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1292 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:40:50.447677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0564b9ef-3d22-4d58-a40e-5e2772343dd9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Action: 'create', calling method: .version_select at 0x7f93098f0310>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-700124936", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCrN3l+jpRTgKlu0lwrjk48hcYVBUfW6rTqLfjWLM7grWDHV+P14GvfYD9pkzgipDPPXKW+U8ARsyknvv+fkD6/2HFfxY40psO8EqNF7toM9uJ32vn0JwNPX+eqrjraZUg=="}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:40:50.462889 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-0564b9ef-3d22-4d58-a40e-5e2772343dd9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Getting quotas for project 372f8b4d0d5c4046ad91bea6d709a6ef. Resources: {'key_pairs'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:40:50.467028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-0564b9ef-3d22-4d58-a40e-5e2772343dd9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Getting quotas for user bf961149e78d4c0c8acb58f1cec447bd and project 372f8b4d0d5c4046ad91bea6d709a6ef. Resources: {'key_pairs'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:40:50.581325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0564b9ef-3d22-4d58-a40e-5e2772343dd9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 399 microversion: 2.1 time: 0.318958 Jun 10 13:40:50.582121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1539/2453] 10.222.0.25 () {60 vars in 1158 bytes} [Fri Jun 10 13:40:50 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:40:50.975240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1aa5ad70-f837-475e-aee8-1d0ba652d5b4 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:50.975834 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1aa5ad70-f837-475e-aee8-1d0ba652d5b4 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:50.983207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1aa5ad70-f837-475e-aee8-1d0ba652d5b4 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:50.983673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1aa5ad70-f837-475e-aee8-1d0ba652d5b4 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:50.984669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1aa5ad70-f837-475e-aee8-1d0ba652d5b4 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:50.985301 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1aa5ad70-f837-475e-aee8-1d0ba652d5b4 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:51.189514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1aa5ad70-f837-475e-aee8-1d0ba652d5b4 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:51.198477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1aa5ad70-f837-475e-aee8-1d0ba652d5b4 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1365 microversion: 2.1 time: 0.226052 Jun 10 13:40:51.213165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1540/2454] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:50 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1365 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:40:52.215582 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7338bc7e-a920-42b1-a1d1-800278db87aa tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:52.217064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7338bc7e-a920-42b1-a1d1-800278db87aa tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:52.224028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7338bc7e-a920-42b1-a1d1-800278db87aa tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:52.224552 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7338bc7e-a920-42b1-a1d1-800278db87aa tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:52.225516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7338bc7e-a920-42b1-a1d1-800278db87aa tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:52.226065 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7338bc7e-a920-42b1-a1d1-800278db87aa tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:52.440558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7338bc7e-a920-42b1-a1d1-800278db87aa tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:52.450604 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7338bc7e-a920-42b1-a1d1-800278db87aa tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1365 microversion: 2.1 time: 0.238704 Jun 10 13:40:52.451490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1541/2455] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:52 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1365 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:40:52.466777 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-99340865-171c-47c1-a082-c008adf0fdaa req-3b2a682d-b6d8-4da3-b776-ae5542fa119d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "73d42484-18bf-4478-8a9d-9da3d4974483", "tag": "81a54bb4-066e-4177-8fa5-494e1a0ee7d0"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:40:52.490818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-99340865-171c-47c1-a082-c008adf0fdaa req-3b2a682d-b6d8-4da3-b776-ae5542fa119d service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:52.492007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-99340865-171c-47c1-a082-c008adf0fdaa req-3b2a682d-b6d8-4da3-b776-ae5542fa119d service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:52.523094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-99340865-171c-47c1-a082-c008adf0fdaa req-3b2a682d-b6d8-4da3-b776-ae5542fa119d service nova] Creating event network-changed:81a54bb4-066e-4177-8fa5-494e1a0ee7d0 for instance 73d42484-18bf-4478-8a9d-9da3d4974483 on hv-cinder-maste Jun 10 13:40:52.529023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-99340865-171c-47c1-a082-c008adf0fdaa req-3b2a682d-b6d8-4da3-b776-ae5542fa119d service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.075993 Jun 10 13:40:52.530218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1542/2456] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:40:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:40:53.472087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-90ccf18f-edb2-4708-9fd7-70d286da8418 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:53.472867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-90ccf18f-edb2-4708-9fd7-70d286da8418 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:53.486073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90ccf18f-edb2-4708-9fd7-70d286da8418 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:53.488277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90ccf18f-edb2-4708-9fd7-70d286da8418 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:53.488277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90ccf18f-edb2-4708-9fd7-70d286da8418 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:53.488277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-90ccf18f-edb2-4708-9fd7-70d286da8418 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:53.893757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-90ccf18f-edb2-4708-9fd7-70d286da8418 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:53.901556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-90ccf18f-edb2-4708-9fd7-70d286da8418 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1365 microversion: 2.1 time: 0.437346 Jun 10 13:40:53.902134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1543/2457] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:53 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1365 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:40:54.920236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0a1f66d0-95c2-443f-bebf-6cc576317085 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:54.921114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0a1f66d0-95c2-443f-bebf-6cc576317085 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:54.928866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=5 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:40:54.930431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0a1f66d0-95c2-443f-bebf-6cc576317085 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:54.930943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0a1f66d0-95c2-443f-bebf-6cc576317085 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:54.932004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0a1f66d0-95c2-443f-bebf-6cc576317085 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:54.932502 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0a1f66d0-95c2-443f-bebf-6cc576317085 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:55.079719 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0a1f66d0-95c2-443f-bebf-6cc576317085 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:55.087934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0a1f66d0-95c2-443f-bebf-6cc576317085 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1365 microversion: 2.1 time: 0.172339 Jun 10 13:40:55.088686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1544/2458] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:54 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1365 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:40:56.105198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c9fe4f67-9be6-4a8b-8df9-6a6f09232e0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:56.105826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c9fe4f67-9be6-4a8b-8df9-6a6f09232e0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:56.113351 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9fe4f67-9be6-4a8b-8df9-6a6f09232e0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:56.113931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9fe4f67-9be6-4a8b-8df9-6a6f09232e0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:56.114690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9fe4f67-9be6-4a8b-8df9-6a6f09232e0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:56.115152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c9fe4f67-9be6-4a8b-8df9-6a6f09232e0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:56.255280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c9fe4f67-9be6-4a8b-8df9-6a6f09232e0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:56.261709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c9fe4f67-9be6-4a8b-8df9-6a6f09232e0f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1365 microversion: 2.1 time: 0.160844 Jun 10 13:40:56.262297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1545/2459] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:56 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1365 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:40:57.283565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-fb70b11a-4f31-4abb-b226-6190472b4b7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:57.284387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-fb70b11a-4f31-4abb-b226-6190472b4b7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:57.292018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb70b11a-4f31-4abb-b226-6190472b4b7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:57.293217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb70b11a-4f31-4abb-b226-6190472b4b7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:57.293394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb70b11a-4f31-4abb-b226-6190472b4b7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:57.293929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-fb70b11a-4f31-4abb-b226-6190472b4b7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:57.392267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:40:57.477393 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-fb70b11a-4f31-4abb-b226-6190472b4b7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:57.485407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-fb70b11a-4f31-4abb-b226-6190472b4b7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1365 microversion: 2.1 time: 0.206974 Jun 10 13:40:57.486001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1546/2460] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:57 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1365 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:40:58.503550 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2d939a42-6672-477d-9181-9b85571a80bd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:58.504225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2d939a42-6672-477d-9181-9b85571a80bd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:58.511243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d939a42-6672-477d-9181-9b85571a80bd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:58.511687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d939a42-6672-477d-9181-9b85571a80bd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:58.512489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d939a42-6672-477d-9181-9b85571a80bd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:58.516369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2d939a42-6672-477d-9181-9b85571a80bd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:58.651195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2d939a42-6672-477d-9181-9b85571a80bd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:58.659278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2d939a42-6672-477d-9181-9b85571a80bd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1365 microversion: 2.1 time: 0.160223 Jun 10 13:40:58.659866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1547/2461] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:58 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1365 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:40:58.890370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-c6b36788-6610-43d2-87ff-b1466a424a7d req-69a02397-7fb6-4cb9-b7c9-0b979515acc8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "73d42484-18bf-4478-8a9d-9da3d4974483", "name": "network-vif-plugged", "status": "completed", "tag": "81a54bb4-066e-4177-8fa5-494e1a0ee7d0"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:40:58.898840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-c6b36788-6610-43d2-87ff-b1466a424a7d req-69a02397-7fb6-4cb9-b7c9-0b979515acc8 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:58.899358 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-c6b36788-6610-43d2-87ff-b1466a424a7d req-69a02397-7fb6-4cb9-b7c9-0b979515acc8 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:58.915822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-c6b36788-6610-43d2-87ff-b1466a424a7d req-69a02397-7fb6-4cb9-b7c9-0b979515acc8 service nova] Creating event network-vif-plugged:81a54bb4-066e-4177-8fa5-494e1a0ee7d0 for instance 73d42484-18bf-4478-8a9d-9da3d4974483 on hv-cinder-maste Jun 10 13:40:58.917949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-c6b36788-6610-43d2-87ff-b1466a424a7d req-69a02397-7fb6-4cb9-b7c9-0b979515acc8 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031120 Jun 10 13:40:58.918714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1548/2462] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:40:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:40:59.676841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-17a37d7a-bee6-4748-b1e6-055695116ba6 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:40:59.677485 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-17a37d7a-bee6-4748-b1e6-055695116ba6 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:40:59.683645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17a37d7a-bee6-4748-b1e6-055695116ba6 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:59.684355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17a37d7a-bee6-4748-b1e6-055695116ba6 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:59.685189 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17a37d7a-bee6-4748-b1e6-055695116ba6 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:40:59.685626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17a37d7a-bee6-4748-b1e6-055695116ba6 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:40:59.822381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-17a37d7a-bee6-4748-b1e6-055695116ba6 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:40:59.831085 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-17a37d7a-bee6-4748-b1e6-055695116ba6 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1365 microversion: 2.1 time: 0.159045 Jun 10 13:40:59.831676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1549/2463] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:40:59 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1365 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:00.848426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e968a15f-9c45-4697-ac5e-18bdfbdd5b9f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:00.849135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e968a15f-9c45-4697-ac5e-18bdfbdd5b9f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:00.850925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=16 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:41:00.856269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e968a15f-9c45-4697-ac5e-18bdfbdd5b9f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:00.856715 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e968a15f-9c45-4697-ac5e-18bdfbdd5b9f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:00.857572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e968a15f-9c45-4697-ac5e-18bdfbdd5b9f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:00.857936 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e968a15f-9c45-4697-ac5e-18bdfbdd5b9f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:00.993396 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e968a15f-9c45-4697-ac5e-18bdfbdd5b9f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:01.002626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e968a15f-9c45-4697-ac5e-18bdfbdd5b9f tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1365 microversion: 2.1 time: 0.157930 Jun 10 13:41:01.003364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1550/2464] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:00 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1365 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:02.020372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2ec15eca-1a00-425e-912b-bd157e2196e3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:02.021086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2ec15eca-1a00-425e-912b-bd157e2196e3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:02.027656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ec15eca-1a00-425e-912b-bd157e2196e3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:02.028105 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ec15eca-1a00-425e-912b-bd157e2196e3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:02.028888 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ec15eca-1a00-425e-912b-bd157e2196e3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:02.029322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2ec15eca-1a00-425e-912b-bd157e2196e3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:02.166173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2ec15eca-1a00-425e-912b-bd157e2196e3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:02.167164 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2ec15eca-1a00-425e-912b-bd157e2196e3 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1554 microversion: 2.1 time: 0.151200 Jun 10 13:41:02.167953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1551/2465] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:02 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1554 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:02.184981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bb951f8b-2adb-4c51-8e9e-cf9713fdceeb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:02.189133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bb951f8b-2adb-4c51-8e9e-cf9713fdceeb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:02.191992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb951f8b-2adb-4c51-8e9e-cf9713fdceeb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:02.192441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb951f8b-2adb-4c51-8e9e-cf9713fdceeb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:02.193259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb951f8b-2adb-4c51-8e9e-cf9713fdceeb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:02.193695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bb951f8b-2adb-4c51-8e9e-cf9713fdceeb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:02.269961 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bb951f8b-2adb-4c51-8e9e-cf9713fdceeb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.089134 Jun 10 13:41:02.270644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1552/2466] 10.222.0.25 () {58 vars in 1274 bytes} [Fri Jun 10 13:41:02 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483/os-interface => generated 284 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:41:03.246816 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-56d0b78d-dc85-4730-9f7e-f360b271b8ac req-9b500972-5617-44d6-b44e-73e9e76e98e9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "73d42484-18bf-4478-8a9d-9da3d4974483", "tag": "81a54bb4-066e-4177-8fa5-494e1a0ee7d0"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:41:03.255669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-56d0b78d-dc85-4730-9f7e-f360b271b8ac req-9b500972-5617-44d6-b44e-73e9e76e98e9 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:03.257648 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-56d0b78d-dc85-4730-9f7e-f360b271b8ac req-9b500972-5617-44d6-b44e-73e9e76e98e9 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:03.267463 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-56d0b78d-dc85-4730-9f7e-f360b271b8ac req-9b500972-5617-44d6-b44e-73e9e76e98e9 service nova] Creating event network-changed:81a54bb4-066e-4177-8fa5-494e1a0ee7d0 for instance 73d42484-18bf-4478-8a9d-9da3d4974483 on hv-cinder-maste Jun 10 13:41:03.272370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-56d0b78d-dc85-4730-9f7e-f360b271b8ac req-9b500972-5617-44d6-b44e-73e9e76e98e9 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029806 Jun 10 13:41:03.273049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1553/2467] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:41:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:41:11.976988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-177510164", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-700124936", "security_groups": [{"name": "tempest-secgroup-smoke-83722939"}], "block_device_mapping_v2": [{"uuid": "724d2817-87cf-4de1-9339-dc0ddcb726c5", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "6241abbb-4625-42fc-b64c-7f8fd8df7647"}]}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:41:12.084449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.network.neutron [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] validate_networks() for [('6241abbb-4625-42fc-b64c-7f8fd8df7647', None, None, None, None, None)] {{(pid=118092) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:41:12.440228 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Flavor limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:41:12.440649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Image limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:41:12.441147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Flavor pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:41:12.441575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Image pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:41:12.441981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:41:12.450775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:12.451216 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:12.451997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:12.452427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:12.469179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Getting quotas for project 372f8b4d0d5c4046ad91bea6d709a6ef. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:41:12.473547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Getting quotas for user bf961149e78d4c0c8acb58f1cec447bd and project 372f8b4d0d5c4046ad91bea6d709a6ef. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:41:12.529893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Going to run 1 instances... {{(pid=118092) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:41:12.532056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='724d2817-87cf-4de1-9339-dc0ddcb726c5',volume_size=None,volume_type=None)] {{(pid=118092) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:41:12.906399 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6dc0766d-4753-4ab9-b4c3-f5dc152ac27e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 0.934394 Jun 10 13:41:12.907052 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1554/2468] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:41:11 2022] POST /compute/v2.1/servers => generated 404 bytes in 936 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jun 10 13:41:12.916832 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-480c497f-d57d-4f05-82c1-156b3d06628d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:12.917442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-480c497f-d57d-4f05-82c1-156b3d06628d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:12.972184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-480c497f-d57d-4f05-82c1-156b3d06628d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:12.979394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-480c497f-d57d-4f05-82c1-156b3d06628d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1093 microversion: 2.1 time: 0.066475 Jun 10 13:41:12.979946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1555/2469] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:12 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1093 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:13.996943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b0b8c61f-7a15-4309-b292-9233f694529c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:13.997567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b0b8c61f-7a15-4309-b292-9233f694529c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:14.004727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b0b8c61f-7a15-4309-b292-9233f694529c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:14.005576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b0b8c61f-7a15-4309-b292-9233f694529c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:14.005949 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b0b8c61f-7a15-4309-b292-9233f694529c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:14.006380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b0b8c61f-7a15-4309-b292-9233f694529c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:14.072394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b0b8c61f-7a15-4309-b292-9233f694529c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:14.079921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b0b8c61f-7a15-4309-b292-9233f694529c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1143 microversion: 2.1 time: 0.087083 Jun 10 13:41:14.081138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1556/2470] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:13 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1143 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:15.100645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6cbb0512-e313-4cca-8752-d2b90678c9f3 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:15.101483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6cbb0512-e313-4cca-8752-d2b90678c9f3 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:15.111121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6cbb0512-e313-4cca-8752-d2b90678c9f3 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:15.111578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6cbb0512-e313-4cca-8752-d2b90678c9f3 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:15.114529 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6cbb0512-e313-4cca-8752-d2b90678c9f3 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:15.114976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6cbb0512-e313-4cca-8752-d2b90678c9f3 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:15.176294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6cbb0512-e313-4cca-8752-d2b90678c9f3 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:15.184073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6cbb0512-e313-4cca-8752-d2b90678c9f3 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1199 microversion: 2.1 time: 0.091073 Jun 10 13:41:15.185161 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1557/2471] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:15 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1199 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:16.203540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-01c3e71d-963d-45ea-9658-6422d350eadf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:16.203540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-01c3e71d-963d-45ea-9658-6422d350eadf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:16.209000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01c3e71d-963d-45ea-9658-6422d350eadf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:16.209444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01c3e71d-963d-45ea-9658-6422d350eadf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:16.210194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01c3e71d-963d-45ea-9658-6422d350eadf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:16.210646 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-01c3e71d-963d-45ea-9658-6422d350eadf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:16.229245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:41:16.341694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-01c3e71d-963d-45ea-9658-6422d350eadf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:16.349955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-01c3e71d-963d-45ea-9658-6422d350eadf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1275 microversion: 2.1 time: 0.152209 Jun 10 13:41:16.350628 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1558/2472] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:16 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1275 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:16.863758 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-f7e90f9c-bc09-4118-ad4d-899453db78df req-8b7a4980-0af9-4fa0-9664-9be17329f032 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8b33753e-2717-477d-b8f7-1f636fee10c4", "tag": "a94adf50-edb8-42cc-9700-56d83642b889"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:41:16.872988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-f7e90f9c-bc09-4118-ad4d-899453db78df req-8b7a4980-0af9-4fa0-9664-9be17329f032 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:16.874200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-f7e90f9c-bc09-4118-ad4d-899453db78df req-8b7a4980-0af9-4fa0-9664-9be17329f032 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:16.885467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-f7e90f9c-bc09-4118-ad4d-899453db78df req-8b7a4980-0af9-4fa0-9664-9be17329f032 service nova] Creating event network-changed:a94adf50-edb8-42cc-9700-56d83642b889 for instance 8b33753e-2717-477d-b8f7-1f636fee10c4 on hv-cinder-maste Jun 10 13:41:16.890710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-f7e90f9c-bc09-4118-ad4d-899453db78df req-8b7a4980-0af9-4fa0-9664-9be17329f032 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031147 Jun 10 13:41:16.891319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1559/2473] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:41:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:41:17.368836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e6fd8bb1-a828-46d8-a2a2-33047860ed8f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:17.369537 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e6fd8bb1-a828-46d8-a2a2-33047860ed8f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:17.376225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e6fd8bb1-a828-46d8-a2a2-33047860ed8f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:17.376618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e6fd8bb1-a828-46d8-a2a2-33047860ed8f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:17.378045 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e6fd8bb1-a828-46d8-a2a2-33047860ed8f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:17.378544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e6fd8bb1-a828-46d8-a2a2-33047860ed8f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:17.530787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e6fd8bb1-a828-46d8-a2a2-33047860ed8f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:17.539044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e6fd8bb1-a828-46d8-a2a2-33047860ed8f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1263 microversion: 2.1 time: 0.174984 Jun 10 13:41:17.539619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1560/2474] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:17 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1263 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:18.556360 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-622a9f8a-83d4-47d8-a7ce-231111480bc7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:18.557060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-622a9f8a-83d4-47d8-a7ce-231111480bc7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:18.563747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-622a9f8a-83d4-47d8-a7ce-231111480bc7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:18.564179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-622a9f8a-83d4-47d8-a7ce-231111480bc7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:18.564974 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-622a9f8a-83d4-47d8-a7ce-231111480bc7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:18.565414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-622a9f8a-83d4-47d8-a7ce-231111480bc7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:18.871068 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-622a9f8a-83d4-47d8-a7ce-231111480bc7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:18.878470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-622a9f8a-83d4-47d8-a7ce-231111480bc7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1263 microversion: 2.1 time: 0.325939 Jun 10 13:41:18.879034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1561/2475] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:18 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1263 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:19.895371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-39e0f562-05ce-4d61-8f56-f88413a27f59 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:19.895995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-39e0f562-05ce-4d61-8f56-f88413a27f59 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:19.901833 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39e0f562-05ce-4d61-8f56-f88413a27f59 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:19.902278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39e0f562-05ce-4d61-8f56-f88413a27f59 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:19.903026 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39e0f562-05ce-4d61-8f56-f88413a27f59 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:19.903455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-39e0f562-05ce-4d61-8f56-f88413a27f59 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:20.029618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-39e0f562-05ce-4d61-8f56-f88413a27f59 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:20.037556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-39e0f562-05ce-4d61-8f56-f88413a27f59 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1263 microversion: 2.1 time: 0.146269 Jun 10 13:41:20.038119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1562/2476] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:19 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1263 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:21.054540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c37712a0-0dd5-41dc-851b-3e3053b2f115 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:21.055137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c37712a0-0dd5-41dc-851b-3e3053b2f115 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:21.061934 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c37712a0-0dd5-41dc-851b-3e3053b2f115 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:21.062375 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c37712a0-0dd5-41dc-851b-3e3053b2f115 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:21.063151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c37712a0-0dd5-41dc-851b-3e3053b2f115 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:21.063578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c37712a0-0dd5-41dc-851b-3e3053b2f115 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:21.192920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c37712a0-0dd5-41dc-851b-3e3053b2f115 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:21.200408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c37712a0-0dd5-41dc-851b-3e3053b2f115 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1263 microversion: 2.1 time: 0.149773 Jun 10 13:41:21.201007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1563/2477] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:21 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1263 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:22.216505 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-14a6131b-6203-4542-bfac-872d267132dd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:22.217219 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-14a6131b-6203-4542-bfac-872d267132dd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:22.224242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-14a6131b-6203-4542-bfac-872d267132dd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:22.224692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-14a6131b-6203-4542-bfac-872d267132dd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:22.225476 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-14a6131b-6203-4542-bfac-872d267132dd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:22.225915 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-14a6131b-6203-4542-bfac-872d267132dd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:22.353143 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-14a6131b-6203-4542-bfac-872d267132dd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:22.360546 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-14a6131b-6203-4542-bfac-872d267132dd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1263 microversion: 2.1 time: 0.147921 Jun 10 13:41:22.361133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1564/2478] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:22 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1263 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:23.378620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ed4c335f-d854-42f5-b0a7-3ac9b048e0fb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:23.379272 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ed4c335f-d854-42f5-b0a7-3ac9b048e0fb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:23.381298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:41:23.382000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=8 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:41:23.387278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ed4c335f-d854-42f5-b0a7-3ac9b048e0fb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:23.387726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ed4c335f-d854-42f5-b0a7-3ac9b048e0fb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:23.388528 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ed4c335f-d854-42f5-b0a7-3ac9b048e0fb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:23.389087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ed4c335f-d854-42f5-b0a7-3ac9b048e0fb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:23.521626 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ed4c335f-d854-42f5-b0a7-3ac9b048e0fb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:23.529012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ed4c335f-d854-42f5-b0a7-3ac9b048e0fb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1263 microversion: 2.1 time: 0.154974 Jun 10 13:41:23.529571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1565/2479] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:23 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1263 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:24.546248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a817cf17-902e-4526-b329-4992e87b2d88 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:24.546932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a817cf17-902e-4526-b329-4992e87b2d88 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:24.553449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a817cf17-902e-4526-b329-4992e87b2d88 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:24.553935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a817cf17-902e-4526-b329-4992e87b2d88 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:24.554784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a817cf17-902e-4526-b329-4992e87b2d88 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:24.555212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a817cf17-902e-4526-b329-4992e87b2d88 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:24.681673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a817cf17-902e-4526-b329-4992e87b2d88 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:24.689276 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a817cf17-902e-4526-b329-4992e87b2d88 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1263 microversion: 2.1 time: 0.147022 Jun 10 13:41:24.689830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1566/2480] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:24 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1263 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:25.707461 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a4852381-7d12-4961-b44f-95675977ece5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:25.708133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a4852381-7d12-4961-b44f-95675977ece5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:25.714864 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a4852381-7d12-4961-b44f-95675977ece5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:25.715305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a4852381-7d12-4961-b44f-95675977ece5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:25.716051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a4852381-7d12-4961-b44f-95675977ece5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:25.716732 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a4852381-7d12-4961-b44f-95675977ece5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:25.845162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a4852381-7d12-4961-b44f-95675977ece5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:25.852588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a4852381-7d12-4961-b44f-95675977ece5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1263 microversion: 2.1 time: 0.149501 Jun 10 13:41:25.853173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1567/2481] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:25 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1263 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:26.870146 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-07f83161-abbf-4c0b-a733-6ba6f4c1c4bf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:26.870839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-07f83161-abbf-4c0b-a733-6ba6f4c1c4bf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:26.873112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=63,nova_api:INSERT=3 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:41:26.878188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07f83161-abbf-4c0b-a733-6ba6f4c1c4bf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:26.878678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07f83161-abbf-4c0b-a733-6ba6f4c1c4bf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:26.879444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07f83161-abbf-4c0b-a733-6ba6f4c1c4bf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:26.879879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07f83161-abbf-4c0b-a733-6ba6f4c1c4bf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:27.018558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-07f83161-abbf-4c0b-a733-6ba6f4c1c4bf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:27.026156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-07f83161-abbf-4c0b-a733-6ba6f4c1c4bf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1263 microversion: 2.1 time: 0.160299 Jun 10 13:41:27.026714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1568/2482] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:26 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1263 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:28.042766 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cb523a38-2acb-4569-8ba5-7b3814fdd2a5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:28.043371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cb523a38-2acb-4569-8ba5-7b3814fdd2a5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:28.050698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb523a38-2acb-4569-8ba5-7b3814fdd2a5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:28.051155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb523a38-2acb-4569-8ba5-7b3814fdd2a5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:28.051926 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb523a38-2acb-4569-8ba5-7b3814fdd2a5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:28.052369 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cb523a38-2acb-4569-8ba5-7b3814fdd2a5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:28.180051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cb523a38-2acb-4569-8ba5-7b3814fdd2a5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:28.187743 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cb523a38-2acb-4569-8ba5-7b3814fdd2a5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1263 microversion: 2.1 time: 0.148820 Jun 10 13:41:28.188307 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1569/2483] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:28 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1263 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:29.037041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-b27fb201-0a63-4875-98a5-993cd15ee6d2 req-a5c7f6b9-99f9-48f7-a8e4-374d5755e45e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8b33753e-2717-477d-b8f7-1f636fee10c4", "name": "network-vif-plugged", "status": "completed", "tag": "a94adf50-edb8-42cc-9700-56d83642b889"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:41:29.045051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-b27fb201-0a63-4875-98a5-993cd15ee6d2 req-a5c7f6b9-99f9-48f7-a8e4-374d5755e45e service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:29.045664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-b27fb201-0a63-4875-98a5-993cd15ee6d2 req-a5c7f6b9-99f9-48f7-a8e4-374d5755e45e service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:29.056593 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-b27fb201-0a63-4875-98a5-993cd15ee6d2 req-a5c7f6b9-99f9-48f7-a8e4-374d5755e45e service nova] Creating event network-vif-plugged:a94adf50-edb8-42cc-9700-56d83642b889 for instance 8b33753e-2717-477d-b8f7-1f636fee10c4 on hv-cinder-maste Jun 10 13:41:29.062370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-b27fb201-0a63-4875-98a5-993cd15ee6d2 req-a5c7f6b9-99f9-48f7-a8e4-374d5755e45e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029133 Jun 10 13:41:29.062960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1570/2484] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:41:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:41:29.205319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-61ffe9e9-9eac-4eba-bfb8-b98a073c26db tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:29.206014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-61ffe9e9-9eac-4eba-bfb8-b98a073c26db tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:29.212229 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61ffe9e9-9eac-4eba-bfb8-b98a073c26db tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:29.212678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61ffe9e9-9eac-4eba-bfb8-b98a073c26db tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:29.213450 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61ffe9e9-9eac-4eba-bfb8-b98a073c26db tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:29.213877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-61ffe9e9-9eac-4eba-bfb8-b98a073c26db tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:29.347049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-61ffe9e9-9eac-4eba-bfb8-b98a073c26db tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:29.354994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-61ffe9e9-9eac-4eba-bfb8-b98a073c26db tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1263 microversion: 2.1 time: 0.154232 Jun 10 13:41:29.358404 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1571/2485] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:29 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1263 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:30.372165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f63b1b19-eb00-4a04-96d3-d879aa7b2add tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:30.372790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f63b1b19-eb00-4a04-96d3-d879aa7b2add tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:30.379335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f63b1b19-eb00-4a04-96d3-d879aa7b2add tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:30.379706 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f63b1b19-eb00-4a04-96d3-d879aa7b2add tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:30.380466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f63b1b19-eb00-4a04-96d3-d879aa7b2add tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:30.382313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f63b1b19-eb00-4a04-96d3-d879aa7b2add tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:30.510845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f63b1b19-eb00-4a04-96d3-d879aa7b2add tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:30.518539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f63b1b19-eb00-4a04-96d3-d879aa7b2add tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1263 microversion: 2.1 time: 0.150194 Jun 10 13:41:30.519647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1572/2486] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:30 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1263 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:31.535818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-29ee99ae-0815-4dbe-9307-b7f91ff2f92b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:31.536459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-29ee99ae-0815-4dbe-9307-b7f91ff2f92b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:31.542881 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-29ee99ae-0815-4dbe-9307-b7f91ff2f92b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:31.543318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-29ee99ae-0815-4dbe-9307-b7f91ff2f92b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:31.544059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-29ee99ae-0815-4dbe-9307-b7f91ff2f92b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:31.544483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-29ee99ae-0815-4dbe-9307-b7f91ff2f92b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:31.673106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-29ee99ae-0815-4dbe-9307-b7f91ff2f92b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:31.681104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-29ee99ae-0815-4dbe-9307-b7f91ff2f92b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1450 microversion: 2.1 time: 0.149107 Jun 10 13:41:31.681664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1573/2487] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:31 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1450 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:31.697010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-20301ede-bf70-40a3-b891-eac55468b718 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:31.697607 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-20301ede-bf70-40a3-b891-eac55468b718 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:31.703075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20301ede-bf70-40a3-b891-eac55468b718 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:31.703510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20301ede-bf70-40a3-b891-eac55468b718 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:31.704257 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20301ede-bf70-40a3-b891-eac55468b718 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:31.704685 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-20301ede-bf70-40a3-b891-eac55468b718 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:31.829493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-20301ede-bf70-40a3-b891-eac55468b718 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:31.836842 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-20301ede-bf70-40a3-b891-eac55468b718 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1450 microversion: 2.1 time: 0.143209 Jun 10 13:41:31.837501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1574/2488] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:31 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1450 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:33.901810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-c9826a77-0912-4ad3-abbf-3943b7797a4e req-f480db75-675f-48ab-a767-2fec44e00936 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8b33753e-2717-477d-b8f7-1f636fee10c4", "tag": "a94adf50-edb8-42cc-9700-56d83642b889"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:41:33.911110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-c9826a77-0912-4ad3-abbf-3943b7797a4e req-f480db75-675f-48ab-a767-2fec44e00936 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:33.911625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-c9826a77-0912-4ad3-abbf-3943b7797a4e req-f480db75-675f-48ab-a767-2fec44e00936 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:33.923060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-c9826a77-0912-4ad3-abbf-3943b7797a4e req-f480db75-675f-48ab-a767-2fec44e00936 service nova] Creating event network-changed:a94adf50-edb8-42cc-9700-56d83642b889 for instance 8b33753e-2717-477d-b8f7-1f636fee10c4 on hv-cinder-maste Jun 10 13:41:33.928748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-c9826a77-0912-4ad3-abbf-3943b7797a4e req-f480db75-675f-48ab-a767-2fec44e00936 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030738 Jun 10 13:41:33.929725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1575/2489] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:41:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:41:39.418604 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-acb37bde-d9d0-4162-a773-d9cf3f201856 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:39.419279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-acb37bde-d9d0-4162-a773-d9cf3f201856 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:39.426519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acb37bde-d9d0-4162-a773-d9cf3f201856 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:39.426959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acb37bde-d9d0-4162-a773-d9cf3f201856 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:39.427704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acb37bde-d9d0-4162-a773-d9cf3f201856 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:39.428129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-acb37bde-d9d0-4162-a773-d9cf3f201856 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:39.450058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-acb37bde-d9d0-4162-a773-d9cf3f201856 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483/ips" status: 200 len: 148 microversion: 2.1 time: 0.035646 Jun 10 13:41:39.450607 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1576/2490] 10.222.0.25 () {58 vars in 1247 bytes} [Fri Jun 10 13:41:39 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483/ips => generated 148 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:41:41.044174 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-63808bb0-e139-4096-8d32-87efbe0bbc0e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "26ef22fe-f7cd-4f17-b83b-3261f54f4e9a"}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:41:41.045280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-63808bb0-e139-4096-8d32-87efbe0bbc0e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:41.051698 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63808bb0-e139-4096-8d32-87efbe0bbc0e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:41.052144 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63808bb0-e139-4096-8d32-87efbe0bbc0e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:41.052917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63808bb0-e139-4096-8d32-87efbe0bbc0e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:41.053355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63808bb0-e139-4096-8d32-87efbe0bbc0e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:41.070108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:41:41.733806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-63808bb0-e139-4096-8d32-87efbe0bbc0e tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "POST /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.693327 Jun 10 13:41:41.735014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1577/2491] 10.222.0.25 () {60 vars in 1322 bytes} [Fri Jun 10 13:41:41 2022] POST /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483/os-volume_attachments => generated 194 bytes in 695 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:41:49.318899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-baed0f4a-3a7c-4538-9c22-b9cd7a5c5492 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:49.319487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-baed0f4a-3a7c-4538-9c22-b9cd7a5c5492 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:49.326252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-baed0f4a-3a7c-4538-9c22-b9cd7a5c5492 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:49.326694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-baed0f4a-3a7c-4538-9c22-b9cd7a5c5492 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:49.327478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-baed0f4a-3a7c-4538-9c22-b9cd7a5c5492 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:49.327909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-baed0f4a-3a7c-4538-9c22-b9cd7a5c5492 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:49.407230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-baed0f4a-3a7c-4538-9c22-b9cd7a5c5492 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lazy-loading 'services' on Instance uuid 73d42484-18bf-4478-8a9d-9da3d4974483 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:41:49.586401 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-baed0f4a-3a7c-4538-9c22-b9cd7a5c5492 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483/os-volume_attachments/26ef22fe-f7cd-4f17-b83b-3261f54f4e9a" status: 202 len: 0 microversion: 2.1 time: 0.271967 Jun 10 13:41:49.586997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1578/2492] 10.222.0.25 () {58 vars in 1415 bytes} [Fri Jun 10 13:41:49 2022] DELETE /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483/os-volume_attachments/26ef22fe-f7cd-4f17-b83b-3261f54f4e9a => generated 0 bytes in 274 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:41:49.595741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dd5fe72a-e29c-43d6-90d7-76d7394c4707 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:49.596893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dd5fe72a-e29c-43d6-90d7-76d7394c4707 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:49.603093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd5fe72a-e29c-43d6-90d7-76d7394c4707 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:49.608893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd5fe72a-e29c-43d6-90d7-76d7394c4707 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:49.608893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd5fe72a-e29c-43d6-90d7-76d7394c4707 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:49.608893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dd5fe72a-e29c-43d6-90d7-76d7394c4707 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:49.634093 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dd5fe72a-e29c-43d6-90d7-76d7394c4707 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042478 Jun 10 13:41:49.634646 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1579/2493] 10.222.0.25 () {58 vars in 1301 bytes} [Fri Jun 10 13:41:49 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:41:50.645782 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2296a47d-43cd-44a1-a8f1-6f5cb5974cfc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:50.646703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2296a47d-43cd-44a1-a8f1-6f5cb5974cfc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:50.653195 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2296a47d-43cd-44a1-a8f1-6f5cb5974cfc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:50.653641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2296a47d-43cd-44a1-a8f1-6f5cb5974cfc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:50.654408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2296a47d-43cd-44a1-a8f1-6f5cb5974cfc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:50.654839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2296a47d-43cd-44a1-a8f1-6f5cb5974cfc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:50.684452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2296a47d-43cd-44a1-a8f1-6f5cb5974cfc tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042575 Jun 10 13:41:50.685058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1580/2494] 10.222.0.25 () {58 vars in 1301 bytes} [Fri Jun 10 13:41:50 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:41:51.696392 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3434cf4a-447f-4d97-a791-b1adcfe901ee tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:51.697342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3434cf4a-447f-4d97-a791-b1adcfe901ee tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:51.703757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3434cf4a-447f-4d97-a791-b1adcfe901ee tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:51.704200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3434cf4a-447f-4d97-a791-b1adcfe901ee tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:51.704993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3434cf4a-447f-4d97-a791-b1adcfe901ee tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:51.705428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3434cf4a-447f-4d97-a791-b1adcfe901ee tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:51.733745 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3434cf4a-447f-4d97-a791-b1adcfe901ee tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041330 Jun 10 13:41:51.734280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1581/2495] 10.222.0.25 () {58 vars in 1301 bytes} [Fri Jun 10 13:41:51 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483/os-volume_attachments => generated 25 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jun 10 13:41:51.791446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-006d109c-f4a4-4067-8765-bb538e0f05cd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:51.795433 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-006d109c-f4a4-4067-8765-bb538e0f05cd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:51.797897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-006d109c-f4a4-4067-8765-bb538e0f05cd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:51.798345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-006d109c-f4a4-4067-8765-bb538e0f05cd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:51.799092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-006d109c-f4a4-4067-8765-bb538e0f05cd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:51.799523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-006d109c-f4a4-4067-8765-bb538e0f05cd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:51.824042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-006d109c-f4a4-4067-8765-bb538e0f05cd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:41:52.135870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-006d109c-f4a4-4067-8765-bb538e0f05cd tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 204 len: 0 microversion: 2.1 time: 0.347834 Jun 10 13:41:52.136380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1582/2496] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:41:51 2022] DELETE /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jun 10 13:41:52.144539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-62cbc71d-2a3c-4315-bef3-0227a4de6f71 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:52.145273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-62cbc71d-2a3c-4315-bef3-0227a4de6f71 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:52.151241 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62cbc71d-2a3c-4315-bef3-0227a4de6f71 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:52.151708 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62cbc71d-2a3c-4315-bef3-0227a4de6f71 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:52.152455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62cbc71d-2a3c-4315-bef3-0227a4de6f71 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:52.152937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-62cbc71d-2a3c-4315-bef3-0227a4de6f71 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:52.283312 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-62cbc71d-2a3c-4315-bef3-0227a4de6f71 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:52.291321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-62cbc71d-2a3c-4315-bef3-0227a4de6f71 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1678 microversion: 2.1 time: 0.150684 Jun 10 13:41:52.291960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1583/2497] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:52 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1678 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:53.308614 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1f412a4c-156b-45ce-aeb3-b9d199786c32 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:53.309364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1f412a4c-156b-45ce-aeb3-b9d199786c32 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:53.315899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1f412a4c-156b-45ce-aeb3-b9d199786c32 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:53.316336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1f412a4c-156b-45ce-aeb3-b9d199786c32 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:53.317740 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1f412a4c-156b-45ce-aeb3-b9d199786c32 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:53.318167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1f412a4c-156b-45ce-aeb3-b9d199786c32 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:53.444104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1f412a4c-156b-45ce-aeb3-b9d199786c32 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:53.451565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1f412a4c-156b-45ce-aeb3-b9d199786c32 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1678 microversion: 2.1 time: 0.146754 Jun 10 13:41:53.452066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1584/2498] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:53 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1678 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:54.468156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-67b03884-4d71-4c82-bc19-bc719949e339 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:54.468917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-67b03884-4d71-4c82-bc19-bc719949e339 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:54.475482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-67b03884-4d71-4c82-bc19-bc719949e339 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:54.475933 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-67b03884-4d71-4c82-bc19-bc719949e339 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:54.476721 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-67b03884-4d71-4c82-bc19-bc719949e339 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:54.477188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-67b03884-4d71-4c82-bc19-bc719949e339 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:54.804047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-67b03884-4d71-4c82-bc19-bc719949e339 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:54.811475 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-67b03884-4d71-4c82-bc19-bc719949e339 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1678 microversion: 2.1 time: 0.347173 Jun 10 13:41:54.812383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1585/2499] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:54 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1678 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:55.409872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-b4de868c-9fbb-4416-b4f9-1abe109f6ae6 req-9a45b8fe-2c14-4687-8f36-a6eec1a74a29 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "73d42484-18bf-4478-8a9d-9da3d4974483", "name": "network-vif-deleted", "tag": "81a54bb4-066e-4177-8fa5-494e1a0ee7d0"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:41:55.419941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-b4de868c-9fbb-4416-b4f9-1abe109f6ae6 req-9a45b8fe-2c14-4687-8f36-a6eec1a74a29 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:55.420407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-b4de868c-9fbb-4416-b4f9-1abe109f6ae6 req-9a45b8fe-2c14-4687-8f36-a6eec1a74a29 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:55.433008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-b4de868c-9fbb-4416-b4f9-1abe109f6ae6 req-9a45b8fe-2c14-4687-8f36-a6eec1a74a29 service nova] Creating event network-vif-deleted:81a54bb4-066e-4177-8fa5-494e1a0ee7d0 for instance 73d42484-18bf-4478-8a9d-9da3d4974483 on hv-cinder-maste Jun 10 13:41:55.436707 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-b4de868c-9fbb-4416-b4f9-1abe109f6ae6 req-9a45b8fe-2c14-4687-8f36-a6eec1a74a29 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030414 Jun 10 13:41:55.437777 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1586/2500] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:41:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:41:55.830608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-552e46e4-93e8-495d-8146-831e18e47f7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:55.831298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-552e46e4-93e8-495d-8146-831e18e47f7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:55.838095 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-552e46e4-93e8-495d-8146-831e18e47f7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:55.838542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-552e46e4-93e8-495d-8146-831e18e47f7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:55.839304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-552e46e4-93e8-495d-8146-831e18e47f7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:55.839739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-552e46e4-93e8-495d-8146-831e18e47f7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:55.870004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-552e46e4-93e8-495d-8146-831e18e47f7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:55.870452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-552e46e4-93e8-495d-8146-831e18e47f7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:55.870901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-552e46e4-93e8-495d-8146-831e18e47f7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lazy-loading 'fault' on Instance uuid 73d42484-18bf-4478-8a9d-9da3d4974483 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:41:55.912213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-552e46e4-93e8-495d-8146-831e18e47f7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '434ae422b68e4145ad8ecac27f450cbc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eee8e4f43f484abdb588a8f3966d74ad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:41:55.919794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-552e46e4-93e8-495d-8146-831e18e47f7d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 200 len: 1330 microversion: 2.1 time: 0.094043 Jun 10 13:41:55.920352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1587/2501] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:55 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 1330 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:41:56.936690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7875b9ec-c9b1-46fd-bb7e-08ddf6b6b73b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:56.937762 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7875b9ec-c9b1-46fd-bb7e-08ddf6b6b73b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:56.945099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7875b9ec-c9b1-46fd-bb7e-08ddf6b6b73b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:56.945544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7875b9ec-c9b1-46fd-bb7e-08ddf6b6b73b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:56.946316 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7875b9ec-c9b1-46fd-bb7e-08ddf6b6b73b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:56.946756 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7875b9ec-c9b1-46fd-bb7e-08ddf6b6b73b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-7875b9ec-c9b1-46fd-bb7e-08ddf6b6b73b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] HTTP exception thrown: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7875b9ec-c9b1-46fd-bb7e-08ddf6b6b73b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Request method failure captured: Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 HTTP/1.1 Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0ftzakwnsjX1_z0cRv3N0JdKbd7XTWcq0bpV9CmZUkNqfVXr0KYvzMAT_PpAbli07mW_6EmovwUniFCsu9Ty1F51kbSG5WbOlXF2lGHoyN4lN1cl-t8ypmG7y3V6y5YPehLSqP2Ng8cCFZlNhxGtZaGC5bhQeMP5jiRYh304eB61I4 Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:41:56.962822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:41:56.965831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_eee8e4f43f484abdb588a8f3966d74ad", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:41:56.965831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:41:56.965831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:41:56.965831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:41:56.965831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:41:56.965831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:41:56.965831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:41:56.965831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 434ae422b68e4145ad8ecac27f450cbc Jun 10 13:41:56.965831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:41:56.965831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:41:56.965831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:56.965831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '73d42484-18bf-4478-8a9d-9da3d4974483'} Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:41:56.966870 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:56.969028 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:56.970558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7875b9ec-c9b1-46fd-bb7e-08ddf6b6b73b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Returning 404 to user: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:41:56.970558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7875b9ec-c9b1-46fd-bb7e-08ddf6b6b73b tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 404 len: 111 microversion: 2.1 time: 0.030699 Jun 10 13:41:56.970558 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1588/2502] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:56 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:41:56.971847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a4deb95e-f535-4e6f-8a0f-df0c2807e159 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '.version_select at 0x7f93052264c0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:56.977546 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=49,nova_api:UPDATE=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:41:57.074365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a4deb95e-f535-4e6f-8a0f-df0c2807e159 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-746089609" status: 202 len: 0 microversion: 2.1 time: 0.106102 Jun 10 13:41:57.074937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1589/2503] 10.222.0.25 () {58 vars in 1217 bytes} [Fri Jun 10 13:41:56 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-746089609 => generated 0 bytes in 107 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jun 10 13:41:59.071370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-887f5eb6-3fa7-41b7-a017-ea4906b63c18 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:59.071952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-887f5eb6-3fa7-41b7-a017-ea4906b63c18 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:59.079117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-887f5eb6-3fa7-41b7-a017-ea4906b63c18 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:59.079575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-887f5eb6-3fa7-41b7-a017-ea4906b63c18 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:59.080343 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-887f5eb6-3fa7-41b7-a017-ea4906b63c18 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:59.080794 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-887f5eb6-3fa7-41b7-a017-ea4906b63c18 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:59.093788 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-887f5eb6-3fa7-41b7-a017-ea4906b63c18 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] HTTP exception thrown: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-887f5eb6-3fa7-41b7-a017-ea4906b63c18 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Request method failure captured: Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 HTTP/1.1 Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0ftzakwnsjX1_z0cRv3N0JdKbd7XTWcq0bpV9CmZUkNqfVXr0KYvzMAT_PpAbli07mW_6EmovwUniFCsu9Ty1F51kbSG5WbOlXF2lGHoyN4lN1cl-t8ypmG7y3V6y5YPehLSqP2Ng8cCFZlNhxGtZaGC5bhQeMP5jiRYh304eB61I4 Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:41:59.095349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:41:59.097157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_eee8e4f43f484abdb588a8f3966d74ad", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:41:59.097157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:41:59.097157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:41:59.097157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:41:59.097157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:41:59.097157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:41:59.097157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:41:59.097157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 434ae422b68e4145ad8ecac27f450cbc Jun 10 13:41:59.097157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:41:59.097157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:41:59.097157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:59.097157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '73d42484-18bf-4478-8a9d-9da3d4974483'} Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:41:59.098744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:59.100394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:59.102167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-887f5eb6-3fa7-41b7-a017-ea4906b63c18 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Returning 404 to user: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:41:59.102167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-887f5eb6-3fa7-41b7-a017-ea4906b63c18 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 404 len: 111 microversion: 2.1 time: 0.029881 Jun 10 13:41:59.102167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1590/2504] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:41:59 2022] DELETE /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:41:59.105377 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6a5078d4-d5e0-4e30-befb-33b4e92d1b9d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:41:59.106115 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6a5078d4-d5e0-4e30-befb-33b4e92d1b9d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: 73d42484-18bf-4478-8a9d-9da3d4974483] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:41:59.111721 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6a5078d4-d5e0-4e30-befb-33b4e92d1b9d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:59.112170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6a5078d4-d5e0-4e30-befb-33b4e92d1b9d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:59.112955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6a5078d4-d5e0-4e30-befb-33b4e92d1b9d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:41:59.113395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6a5078d4-d5e0-4e30-befb-33b4e92d1b9d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:41:59.127089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-6a5078d4-d5e0-4e30-befb-33b4e92d1b9d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] HTTP exception thrown: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6a5078d4-d5e0-4e30-befb-33b4e92d1b9d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Request method failure captured: Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 HTTP/1.1 Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0ftzakwnsjX1_z0cRv3N0JdKbd7XTWcq0bpV9CmZUkNqfVXr0KYvzMAT_PpAbli07mW_6EmovwUniFCsu9Ty1F51kbSG5WbOlXF2lGHoyN4lN1cl-t8ypmG7y3V6y5YPehLSqP2Ng8cCFZlNhxGtZaGC5bhQeMP5jiRYh304eB61I4 Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:41:59.128872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:41:59.130659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_eee8e4f43f484abdb588a8f3966d74ad", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:41:59.130659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:41:59.130659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:41:59.130659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:41:59.130659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:41:59.130659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:41:59.130659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:41:59.130659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 434ae422b68e4145ad8ecac27f450cbc Jun 10 13:41:59.130659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:41:59.130659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:41:59.130659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:59.130659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '73d42484-18bf-4478-8a9d-9da3d4974483'} Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:41:59.131828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. Jun 10 13:41:59.133659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:41:59.135238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6a5078d4-d5e0-4e30-befb-33b4e92d1b9d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Returning 404 to user: Instance 73d42484-18bf-4478-8a9d-9da3d4974483 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:41:59.135238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6a5078d4-d5e0-4e30-befb-33b4e92d1b9d tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483" status: 404 len: 111 microversion: 2.1 time: 0.028419 Jun 10 13:41:59.135238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1591/2505] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:41:59 2022] GET /compute/v2.1/servers/73d42484-18bf-4478-8a9d-9da3d4974483 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:42:00.393839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-75bbc2f3-6507-4037-9fac-57cf7f0a5cbb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:42:00.394423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-75bbc2f3-6507-4037-9fac-57cf7f0a5cbb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:42:00.400889 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75bbc2f3-6507-4037-9fac-57cf7f0a5cbb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:42:00.401338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75bbc2f3-6507-4037-9fac-57cf7f0a5cbb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:42:00.402096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75bbc2f3-6507-4037-9fac-57cf7f0a5cbb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:42:00.402526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-75bbc2f3-6507-4037-9fac-57cf7f0a5cbb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:42:00.416012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-75bbc2f3-6507-4037-9fac-57cf7f0a5cbb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] HTTP exception thrown: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-75bbc2f3-6507-4037-9fac-57cf7f0a5cbb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Request method failure captured: Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 HTTP/1.1 Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0ftzakwnsjX1_z0cRv3N0JdKbd7XTWcq0bpV9CmZUkNqfVXr0KYvzMAT_PpAbli07mW_6EmovwUniFCsu9Ty1F51kbSG5WbOlXF2lGHoyN4lN1cl-t8ypmG7y3V6y5YPehLSqP2Ng8cCFZlNhxGtZaGC5bhQeMP5jiRYh304eB61I4 Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:42:00.417565 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:42:00.419330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_eee8e4f43f484abdb588a8f3966d74ad", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:42:00.419330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:42:00.419330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:42:00.419330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:42:00.419330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:42:00.419330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:42:00.419330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:42:00.419330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 434ae422b68e4145ad8ecac27f450cbc Jun 10 13:42:00.419330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:42:00.419330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:42:00.419330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:42:00.419330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'fca751de-4ecc-444e-af32-ea1b33e262e7'} Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:42:00.420345 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:42:00.422042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:00.423966 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-75bbc2f3-6507-4037-9fac-57cf7f0a5cbb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Returning 404 to user: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:42:00.423966 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-75bbc2f3-6507-4037-9fac-57cf7f0a5cbb tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 404 len: 111 microversion: 2.1 time: 0.028946 Jun 10 13:42:00.423966 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1592/2506] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:42:00 2022] DELETE /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:42:00.427272 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2022550b-fa5e-4843-9dbe-9e17125c56c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:42:00.427887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2022550b-fa5e-4843-9dbe-9e17125c56c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: fca751de-4ecc-444e-af32-ea1b33e262e7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:42:00.433470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2022550b-fa5e-4843-9dbe-9e17125c56c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:42:00.433918 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2022550b-fa5e-4843-9dbe-9e17125c56c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:42:00.434659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2022550b-fa5e-4843-9dbe-9e17125c56c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:42:00.435167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2022550b-fa5e-4843-9dbe-9e17125c56c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:42:00.448240 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-2022550b-fa5e-4843-9dbe-9e17125c56c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] HTTP exception thrown: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2022550b-fa5e-4843-9dbe-9e17125c56c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Request method failure captured: Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 HTTP/1.1 Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0ftzakwnsjX1_z0cRv3N0JdKbd7XTWcq0bpV9CmZUkNqfVXr0KYvzMAT_PpAbli07mW_6EmovwUniFCsu9Ty1F51kbSG5WbOlXF2lGHoyN4lN1cl-t8ypmG7y3V6y5YPehLSqP2Ng8cCFZlNhxGtZaGC5bhQeMP5jiRYh304eB61I4 Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:42:00.449753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:42:00.451579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_eee8e4f43f484abdb588a8f3966d74ad", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:42:00.451579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:42:00.451579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:42:00.451579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:42:00.451579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:42:00.451579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:42:00.451579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:42:00.451579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 434ae422b68e4145ad8ecac27f450cbc Jun 10 13:42:00.451579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:42:00.451579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:42:00.451579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:42:00.451579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'fca751de-4ecc-444e-af32-ea1b33e262e7'} Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:42:00.452635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. Jun 10 13:42:00.454313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:00.456111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2022550b-fa5e-4843-9dbe-9e17125c56c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Returning 404 to user: Instance fca751de-4ecc-444e-af32-ea1b33e262e7 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:42:00.456111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2022550b-fa5e-4843-9dbe-9e17125c56c0 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7" status: 404 len: 111 microversion: 2.1 time: 0.027843 Jun 10 13:42:00.456111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1593/2507] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:42:00 2022] GET /compute/v2.1/servers/fca751de-4ecc-444e-af32-ea1b33e262e7 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:42:01.713302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-71ee462b-64d4-4fdf-91e1-cdece0db7e40 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:42:01.713841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-71ee462b-64d4-4fdf-91e1-cdece0db7e40 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:42:01.720083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71ee462b-64d4-4fdf-91e1-cdece0db7e40 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:42:01.720523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71ee462b-64d4-4fdf-91e1-cdece0db7e40 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:42:01.721302 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71ee462b-64d4-4fdf-91e1-cdece0db7e40 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:42:01.721731 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-71ee462b-64d4-4fdf-91e1-cdece0db7e40 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:42:01.736855 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-71ee462b-64d4-4fdf-91e1-cdece0db7e40 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] HTTP exception thrown: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-71ee462b-64d4-4fdf-91e1-cdece0db7e40 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Request method failure captured: Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 HTTP/1.1 Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0ftzakwnsjX1_z0cRv3N0JdKbd7XTWcq0bpV9CmZUkNqfVXr0KYvzMAT_PpAbli07mW_6EmovwUniFCsu9Ty1F51kbSG5WbOlXF2lGHoyN4lN1cl-t8ypmG7y3V6y5YPehLSqP2Ng8cCFZlNhxGtZaGC5bhQeMP5jiRYh304eB61I4 Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:42:01.738366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:42:01.740757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_eee8e4f43f484abdb588a8f3966d74ad", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:42:01.740757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:42:01.740757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:42:01.740757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:42:01.740757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:42:01.740757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:42:01.740757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:42:01.740757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 434ae422b68e4145ad8ecac27f450cbc Jun 10 13:42:01.740757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:42:01.740757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:42:01.740757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:42:01.740757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'b923ce4b-4493-492c-82b8-4adc45ea47f7'} Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:42:01.742024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:42:01.743789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:01.745496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-71ee462b-64d4-4fdf-91e1-cdece0db7e40 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Returning 404 to user: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:42:01.745496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-71ee462b-64d4-4fdf-91e1-cdece0db7e40 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 404 len: 111 microversion: 2.1 time: 0.030678 Jun 10 13:42:01.745496 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1594/2508] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:42:01 2022] DELETE /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:42:01.747929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b7988ad2-c8ed-4296-b77c-984ff288a9f7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:42:01.748519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b7988ad2-c8ed-4296-b77c-984ff288a9f7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] [instance: b923ce4b-4493-492c-82b8-4adc45ea47f7] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:42:01.754243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b7988ad2-c8ed-4296-b77c-984ff288a9f7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:42:01.755249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b7988ad2-c8ed-4296-b77c-984ff288a9f7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:42:01.755990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b7988ad2-c8ed-4296-b77c-984ff288a9f7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:42:01.756417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b7988ad2-c8ed-4296-b77c-984ff288a9f7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:42:01.769209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-b7988ad2-c8ed-4296-b77c-984ff288a9f7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] HTTP exception thrown: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b7988ad2-c8ed-4296-b77c-984ff288a9f7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Request method failure captured: Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 HTTP/1.1 Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0ftzakwnsjX1_z0cRv3N0JdKbd7XTWcq0bpV9CmZUkNqfVXr0KYvzMAT_PpAbli07mW_6EmovwUniFCsu9Ty1F51kbSG5WbOlXF2lGHoyN4lN1cl-t8ypmG7y3V6y5YPehLSqP2Ng8cCFZlNhxGtZaGC5bhQeMP5jiRYh304eB61I4 Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:42:01.770744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:42:01.772517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_eee8e4f43f484abdb588a8f3966d74ad", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/eee8e4f43f484abdb588a8f3966d74ad"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:42:01.772517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:42:01.772517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: eee8e4f43f484abdb588a8f3966d74ad Jun 10 13:42:01.772517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1732915129 Jun 10 13:42:01.772517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:42:01.772517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:42:01.772517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:42:01.772517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 434ae422b68e4145ad8ecac27f450cbc Jun 10 13:42:01.772517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeNegativeTest-1732915129-project Jun 10 13:42:01.772517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:42:01.772517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:42:01.772517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'b923ce4b-4493-492c-82b8-4adc45ea47f7'} Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:42:01.773727 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. Jun 10 13:42:01.775356 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:01.776960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b7988ad2-c8ed-4296-b77c-984ff288a9f7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] Returning 404 to user: Instance b923ce4b-4493-492c-82b8-4adc45ea47f7 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:42:01.776960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b7988ad2-c8ed-4296-b77c-984ff288a9f7 tempest-AttachVolumeNegativeTest-1732915129 tempest-AttachVolumeNegativeTest-1732915129-project] 10.222.0.25 "GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7" status: 404 len: 111 microversion: 2.1 time: 0.027522 Jun 10 13:42:01.776960 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1595/2509] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:42:01 2022] GET /compute/v2.1/servers/b923ce4b-4493-492c-82b8-4adc45ea47f7 => generated 111 bytes in 28 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:42:22.627578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f398f573-90e1-44ba-9a49-cb8998fb6ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:42:22.628157 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f398f573-90e1-44ba-9a49-cb8998fb6ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:42:22.630192 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=14,nova_api:DELETE=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:42:22.630872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:42:22.636250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f398f573-90e1-44ba-9a49-cb8998fb6ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:42:22.636695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f398f573-90e1-44ba-9a49-cb8998fb6ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:42:22.637487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f398f573-90e1-44ba-9a49-cb8998fb6ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:42:22.637969 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f398f573-90e1-44ba-9a49-cb8998fb6ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:42:22.652601 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-f398f573-90e1-44ba-9a49-cb8998fb6ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] HTTP exception thrown: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f398f573-90e1-44ba-9a49-cb8998fb6ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Request method failure captured: Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 HTTP/1.1 Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0GI4neGqGl6u4NtxR88mPQrCvR_v1VmiIEaXPKEqlwdyaT_vIeeEJpWEM0993X6W9IucjryiBRG6JJNHEx9ulNG1lmVs69VgCwa572eOjYLN6_EoRIIXPkdDj4Zac85isYH3jNfGblme5mU9zzCrvjMFkL5ny73zNQIxIWZatGPexU Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Openstack-Nova-Api-Version: 2.20 Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 28f8244decff453e98e2d372a97a563c Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:42:22.654186 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:42:22.656099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/28f8244decff453e98e2d372a97a563c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/28f8244decff453e98e2d372a97a563c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_28f8244decff453e98e2d372a97a563c", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/28f8244decff453e98e2d372a97a563c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:42:22.656099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:42:22.656099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 28f8244decff453e98e2d372a97a563c Jun 10 13:42:22.656099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:42:22.656099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeShelveTestJSON-1465745724-project Jun 10 13:42:22.656099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:42:22.656099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:42:22.656099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: b1d1712ea523498f92f258383863b906 Jun 10 13:42:22.656099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1465745724-project Jun 10 13:42:22.656099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:42:22.656099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:42:22.656099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'a99c1a73-3f56-4c71-a4fd-67988bbb7464'} Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:42:22.657018 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:42:22.659039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:22.660690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f398f573-90e1-44ba-9a49-cb8998fb6ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Returning 404 to user: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:42:22.660690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f398f573-90e1-44ba-9a49-cb8998fb6ba9 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "DELETE /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 404 len: 111 microversion: 2.20 time: 0.032000 Jun 10 13:42:22.660690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1596/2510] 10.222.0.25 () {60 vars in 1279 bytes} [Fri Jun 10 13:42:22 2022] DELETE /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jun 10 13:42:22.663883 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-634e6cc5-a977-499a-8ded-88cdb8465ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:42:22.664498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-634e6cc5-a977-499a-8ded-88cdb8465ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: a99c1a73-3f56-4c71-a4fd-67988bbb7464] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:42:22.670943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-634e6cc5-a977-499a-8ded-88cdb8465ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:42:22.671377 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-634e6cc5-a977-499a-8ded-88cdb8465ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:42:22.672154 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-634e6cc5-a977-499a-8ded-88cdb8465ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:42:22.672584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-634e6cc5-a977-499a-8ded-88cdb8465ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:42:22.687481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-634e6cc5-a977-499a-8ded-88cdb8465ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] HTTP exception thrown: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-634e6cc5-a977-499a-8ded-88cdb8465ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Request method failure captured: Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 HTTP/1.1 Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0GI4neGqGl6u4NtxR88mPQrCvR_v1VmiIEaXPKEqlwdyaT_vIeeEJpWEM0993X6W9IucjryiBRG6JJNHEx9ulNG1lmVs69VgCwa572eOjYLN6_EoRIIXPkdDj4Zac85isYH3jNfGblme5mU9zzCrvjMFkL5ny73zNQIxIWZatGPexU Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Openstack-Nova-Api-Version: 2.20 Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 28f8244decff453e98e2d372a97a563c Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:42:22.689317 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:42:22.691177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/28f8244decff453e98e2d372a97a563c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/28f8244decff453e98e2d372a97a563c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_28f8244decff453e98e2d372a97a563c", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/28f8244decff453e98e2d372a97a563c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:42:22.691177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:42:22.691177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 28f8244decff453e98e2d372a97a563c Jun 10 13:42:22.691177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:42:22.691177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeShelveTestJSON-1465745724-project Jun 10 13:42:22.691177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:42:22.691177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:42:22.691177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: b1d1712ea523498f92f258383863b906 Jun 10 13:42:22.691177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1465745724-project Jun 10 13:42:22.691177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:42:22.691177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:42:22.691177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'a99c1a73-3f56-4c71-a4fd-67988bbb7464'} Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:42:22.692380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. Jun 10 13:42:22.694193 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:42:22.696332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-634e6cc5-a977-499a-8ded-88cdb8465ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Returning 404 to user: Instance a99c1a73-3f56-4c71-a4fd-67988bbb7464 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:42:22.696332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-634e6cc5-a977-499a-8ded-88cdb8465ec3 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464" status: 404 len: 111 microversion: 2.20 time: 0.030532 Jun 10 13:42:22.696332 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1597/2511] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:42:22 2022] GET /compute/v2.1/servers/a99c1a73-3f56-4c71-a4fd-67988bbb7464 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jun 10 13:43:36.215879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=4 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:43:36.219767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a0953add-c155-47ea-a7b5-24b08884eb8b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:36.220264 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a0953add-c155-47ea-a7b5-24b08884eb8b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:36.227871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0953add-c155-47ea-a7b5-24b08884eb8b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:36.227871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0953add-c155-47ea-a7b5-24b08884eb8b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:36.227871 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0953add-c155-47ea-a7b5-24b08884eb8b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:36.228835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a0953add-c155-47ea-a7b5-24b08884eb8b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:36.254659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a0953add-c155-47ea-a7b5-24b08884eb8b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:43:36.512865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a0953add-c155-47ea-a7b5-24b08884eb8b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "DELETE /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 204 len: 0 microversion: 2.1 time: 0.298753 Jun 10 13:43:36.515088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1598/2512] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:43:36 2022] DELETE /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jun 10 13:43:36.522049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-731e24b3-2897-475f-8ddd-853593e38a89 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:36.522752 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-731e24b3-2897-475f-8ddd-853593e38a89 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:36.528845 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-731e24b3-2897-475f-8ddd-853593e38a89 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:36.529296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-731e24b3-2897-475f-8ddd-853593e38a89 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:36.530059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-731e24b3-2897-475f-8ddd-853593e38a89 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:36.530488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-731e24b3-2897-475f-8ddd-853593e38a89 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:36.662709 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-731e24b3-2897-475f-8ddd-853593e38a89 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:36.671075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-731e24b3-2897-475f-8ddd-853593e38a89 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1575 microversion: 2.1 time: 0.152958 Jun 10 13:43:36.671633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1599/2513] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:36 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1575 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:37.687921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8ba90f65-3c0f-47f4-976d-1fa5cbf81cfc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:37.688575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8ba90f65-3c0f-47f4-976d-1fa5cbf81cfc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:37.695314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ba90f65-3c0f-47f4-976d-1fa5cbf81cfc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:37.695760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ba90f65-3c0f-47f4-976d-1fa5cbf81cfc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:37.696512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ba90f65-3c0f-47f4-976d-1fa5cbf81cfc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:37.696979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8ba90f65-3c0f-47f4-976d-1fa5cbf81cfc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:37.831858 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8ba90f65-3c0f-47f4-976d-1fa5cbf81cfc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:37.839572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8ba90f65-3c0f-47f4-976d-1fa5cbf81cfc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1575 microversion: 2.1 time: 0.155973 Jun 10 13:43:37.840134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1600/2514] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:37 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1575 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:38.857557 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-00cb6f2f-ba3e-4435-8cb9-728ef02c638c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:38.858364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-00cb6f2f-ba3e-4435-8cb9-728ef02c638c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:38.865814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00cb6f2f-ba3e-4435-8cb9-728ef02c638c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:38.866645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00cb6f2f-ba3e-4435-8cb9-728ef02c638c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:38.867928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00cb6f2f-ba3e-4435-8cb9-728ef02c638c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:38.868365 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-00cb6f2f-ba3e-4435-8cb9-728ef02c638c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:39.084943 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-00cb6f2f-ba3e-4435-8cb9-728ef02c638c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:39.094145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-00cb6f2f-ba3e-4435-8cb9-728ef02c638c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1575 microversion: 2.1 time: 0.241133 Jun 10 13:43:39.098080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1601/2515] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:38 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1575 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:39.543175 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-c8c04af1-b2fd-424e-8015-1d247fc109eb req-373fae6c-590f-4a39-ab79-a46254b660ee service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8b33753e-2717-477d-b8f7-1f636fee10c4", "name": "network-vif-deleted", "tag": "a94adf50-edb8-42cc-9700-56d83642b889"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:43:39.544500 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-c8c04af1-b2fd-424e-8015-1d247fc109eb req-373fae6c-590f-4a39-ab79-a46254b660ee service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:39.544979 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-c8c04af1-b2fd-424e-8015-1d247fc109eb req-373fae6c-590f-4a39-ab79-a46254b660ee service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:39.557392 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-c8c04af1-b2fd-424e-8015-1d247fc109eb req-373fae6c-590f-4a39-ab79-a46254b660ee service nova] Creating event network-vif-deleted:a94adf50-edb8-42cc-9700-56d83642b889 for instance 8b33753e-2717-477d-b8f7-1f636fee10c4 on hv-cinder-maste Jun 10 13:43:39.562566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-c8c04af1-b2fd-424e-8015-1d247fc109eb req-373fae6c-590f-4a39-ab79-a46254b660ee service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029411 Jun 10 13:43:39.563159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1602/2516] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:43:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:43:40.112884 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0c21cc82-4011-4c28-aae3-f32faade5ae7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:40.113519 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0c21cc82-4011-4c28-aae3-f32faade5ae7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:40.119801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c21cc82-4011-4c28-aae3-f32faade5ae7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:40.120269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c21cc82-4011-4c28-aae3-f32faade5ae7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:40.121774 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c21cc82-4011-4c28-aae3-f32faade5ae7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:40.122215 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0c21cc82-4011-4c28-aae3-f32faade5ae7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:40.184171 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0c21cc82-4011-4c28-aae3-f32faade5ae7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:40.192047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0c21cc82-4011-4c28-aae3-f32faade5ae7 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1224 microversion: 2.1 time: 0.083819 Jun 10 13:43:40.192913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1603/2517] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:40 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:41.208831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e77be49e-56aa-4be0-b2b1-e0bb2757b6bc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:41.209982 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e77be49e-56aa-4be0-b2b1-e0bb2757b6bc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:41.217172 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e77be49e-56aa-4be0-b2b1-e0bb2757b6bc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:41.217994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e77be49e-56aa-4be0-b2b1-e0bb2757b6bc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:41.218757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e77be49e-56aa-4be0-b2b1-e0bb2757b6bc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:41.219214 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e77be49e-56aa-4be0-b2b1-e0bb2757b6bc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:41.250516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e77be49e-56aa-4be0-b2b1-e0bb2757b6bc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:41.250964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e77be49e-56aa-4be0-b2b1-e0bb2757b6bc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:41.251414 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-e77be49e-56aa-4be0-b2b1-e0bb2757b6bc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lazy-loading 'fault' on Instance uuid 8b33753e-2717-477d-b8f7-1f636fee10c4 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:43:41.291155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e77be49e-56aa-4be0-b2b1-e0bb2757b6bc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:41.300194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e77be49e-56aa-4be0-b2b1-e0bb2757b6bc tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 200 len: 1183 microversion: 2.1 time: 0.095299 Jun 10 13:43:41.300801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1604/2518] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:41 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 1183 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:42.316905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c1507b4a-cb2c-458b-b21a-008217074cbd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:42.317795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c1507b4a-cb2c-458b-b21a-008217074cbd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:42.324089 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1507b4a-cb2c-458b-b21a-008217074cbd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:42.327000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1507b4a-cb2c-458b-b21a-008217074cbd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:42.327000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1507b4a-cb2c-458b-b21a-008217074cbd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:42.327000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c1507b4a-cb2c-458b-b21a-008217074cbd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:42.339070 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-c1507b4a-cb2c-458b-b21a-008217074cbd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] HTTP exception thrown: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c1507b4a-cb2c-458b-b21a-008217074cbd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Request method failure captured: Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 HTTP/1.1 Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0naMDwEZ-YRSB7u9MWmuxa7VCZvbw5ekhxDtysZCwuiejBDwGfJK6HGQuuDfJjuNsnxxtAcrbNyuDjyZEn2ArGkghDHQNGqM2deKRpOfTSFnRaPrcSQSdBg7vhdrLqh7OJInyc42VEgwet4IzwTw2sa1gRaoreRqelNgRh5zNivNGE Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 372f8b4d0d5c4046ad91bea6d709a6ef Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:43:42.340714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:43:42.342644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_372f8b4d0d5c4046ad91bea6d709a6ef", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:43:42.342644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:43:42.342644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 372f8b4d0d5c4046ad91bea6d709a6ef Jun 10 13:43:42.342644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:43:42.342644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-TestVolumeBootPattern-1295034315-project Jun 10 13:43:42.342644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:43:42.342644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:43:42.342644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: bf961149e78d4c0c8acb58f1cec447bd Jun 10 13:43:42.342644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-TestVolumeBootPattern-1295034315-project Jun 10 13:43:42.342644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:43:42.342644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:43:42.342644 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '8b33753e-2717-477d-b8f7-1f636fee10c4'} Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:43:42.343599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:43:42.345326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:43:42.346643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c1507b4a-cb2c-458b-b21a-008217074cbd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Returning 404 to user: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:43:42.346643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c1507b4a-cb2c-458b-b21a-008217074cbd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 404 len: 111 microversion: 2.1 time: 0.029150 Jun 10 13:43:42.346643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1605/2519] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:42 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:43:42.436579 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1739085619", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-700124936", "security_groups": [{"name": "tempest-secgroup-smoke-83722939"}], "block_device_mapping_v2": [{"uuid": "724d2817-87cf-4de1-9339-dc0ddcb726c5", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "6241abbb-4625-42fc-b64c-7f8fd8df7647"}]}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:43:42.547067 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.network.neutron [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] validate_networks() for [('6241abbb-4625-42fc-b64c-7f8fd8df7647', None, None, None, None, None)] {{(pid=118092) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:43:42.914279 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Flavor limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:43:42.914776 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Image limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:43:42.915271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Flavor pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:43:42.915729 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Image pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:43:42.916138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:43:42.926048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:42.926489 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:42.927268 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:42.927702 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:42.944986 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Getting quotas for project 372f8b4d0d5c4046ad91bea6d709a6ef. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:43:42.948988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Getting quotas for user bf961149e78d4c0c8acb58f1cec447bd and project 372f8b4d0d5c4046ad91bea6d709a6ef. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:43:43.005972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Going to run 1 instances... {{(pid=118092) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:43:43.008047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='724d2817-87cf-4de1-9339-dc0ddcb726c5',volume_size=None,volume_type=None)] {{(pid=118092) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:43:43.419542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8eb47c18-4ae5-4831-9dce-5169090bda32 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 0.987147 Jun 10 13:43:43.425584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1606/2520] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:43:42 2022] POST /compute/v2.1/servers => generated 404 bytes in 994 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jun 10 13:43:43.431368 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e7fe3bb7-fe5b-4120-93d2-b9d386a3bc9c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:43.432064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e7fe3bb7-fe5b-4120-93d2-b9d386a3bc9c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:43.490780 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e7fe3bb7-fe5b-4120-93d2-b9d386a3bc9c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:43.499512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e7fe3bb7-fe5b-4120-93d2-b9d386a3bc9c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1094 microversion: 2.1 time: 0.072354 Jun 10 13:43:43.502481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1607/2521] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:43 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1094 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:43:44.536060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7f020e3b-7333-4e62-adbc-65ee1a8b6251 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:44.537421 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7f020e3b-7333-4e62-adbc-65ee1a8b6251 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:44.552530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f020e3b-7333-4e62-adbc-65ee1a8b6251 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:44.553008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f020e3b-7333-4e62-adbc-65ee1a8b6251 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:44.553785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f020e3b-7333-4e62-adbc-65ee1a8b6251 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:44.554218 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7f020e3b-7333-4e62-adbc-65ee1a8b6251 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:44.619876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7f020e3b-7333-4e62-adbc-65ee1a8b6251 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:44.628254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7f020e3b-7333-4e62-adbc-65ee1a8b6251 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1144 microversion: 2.1 time: 0.096365 Jun 10 13:43:44.638641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1608/2522] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:44 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1144 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:45.646135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bcba48b4-d2f4-4735-89ec-aee3edc2b465 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:45.648087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bcba48b4-d2f4-4735-89ec-aee3edc2b465 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:45.660278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bcba48b4-d2f4-4735-89ec-aee3edc2b465 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:45.660278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bcba48b4-d2f4-4735-89ec-aee3edc2b465 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:45.660278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bcba48b4-d2f4-4735-89ec-aee3edc2b465 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:45.660278 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bcba48b4-d2f4-4735-89ec-aee3edc2b465 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:45.757754 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bcba48b4-d2f4-4735-89ec-aee3edc2b465 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:45.770108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bcba48b4-d2f4-4735-89ec-aee3edc2b465 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1200 microversion: 2.1 time: 0.127572 Jun 10 13:43:45.773249 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1609/2523] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:45 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1200 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:46.788887 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2554ef0c-d65d-4c1f-9899-c02d61b37841 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:46.790191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2554ef0c-d65d-4c1f-9899-c02d61b37841 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:46.798096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2554ef0c-d65d-4c1f-9899-c02d61b37841 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:46.798799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2554ef0c-d65d-4c1f-9899-c02d61b37841 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:46.799942 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2554ef0c-d65d-4c1f-9899-c02d61b37841 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:46.800678 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2554ef0c-d65d-4c1f-9899-c02d61b37841 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:47.115611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2554ef0c-d65d-4c1f-9899-c02d61b37841 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:47.138680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2554ef0c-d65d-4c1f-9899-c02d61b37841 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1276 microversion: 2.1 time: 0.344097 Jun 10 13:43:47.142012 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1610/2524] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:46 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1276 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:48.045627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-14528cd3-87ec-49fb-ac29-c4f41f1d27cb req-0c6e787e-7ae5-411c-a58c-53148c21274b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "53f34c67-563c-4a95-ac23-25e7d05d305d", "tag": "c60dc47c-8f9b-48bf-b642-4c2f82a01fe6"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:43:48.065182 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-14528cd3-87ec-49fb-ac29-c4f41f1d27cb req-0c6e787e-7ae5-411c-a58c-53148c21274b service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:48.065620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-14528cd3-87ec-49fb-ac29-c4f41f1d27cb req-0c6e787e-7ae5-411c-a58c-53148c21274b service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:48.088608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-14528cd3-87ec-49fb-ac29-c4f41f1d27cb req-0c6e787e-7ae5-411c-a58c-53148c21274b service nova] Creating event network-changed:c60dc47c-8f9b-48bf-b642-4c2f82a01fe6 for instance 53f34c67-563c-4a95-ac23-25e7d05d305d on hv-cinder-maste Jun 10 13:43:48.103762 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-14528cd3-87ec-49fb-ac29-c4f41f1d27cb req-0c6e787e-7ae5-411c-a58c-53148c21274b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060825 Jun 10 13:43:48.104440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1611/2525] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:43:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:43:48.158138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-167a8375-7370-48fd-b27a-830f7da2fc27 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:48.158748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-167a8375-7370-48fd-b27a-830f7da2fc27 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:48.173813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-167a8375-7370-48fd-b27a-830f7da2fc27 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:48.174540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-167a8375-7370-48fd-b27a-830f7da2fc27 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:48.175978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-167a8375-7370-48fd-b27a-830f7da2fc27 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:48.176792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-167a8375-7370-48fd-b27a-830f7da2fc27 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:48.452835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-167a8375-7370-48fd-b27a-830f7da2fc27 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:48.461652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-167a8375-7370-48fd-b27a-830f7da2fc27 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1264 microversion: 2.1 time: 0.307936 Jun 10 13:43:48.462113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1612/2526] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:48 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1264 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:49.479148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7ea801a0-06a7-43ab-bdcd-24ee4a39d546 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:49.479805 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7ea801a0-06a7-43ab-bdcd-24ee4a39d546 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:49.486806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ea801a0-06a7-43ab-bdcd-24ee4a39d546 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:49.487259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ea801a0-06a7-43ab-bdcd-24ee4a39d546 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:49.488060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ea801a0-06a7-43ab-bdcd-24ee4a39d546 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:49.488492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7ea801a0-06a7-43ab-bdcd-24ee4a39d546 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:49.634831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7ea801a0-06a7-43ab-bdcd-24ee4a39d546 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:49.642905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7ea801a0-06a7-43ab-bdcd-24ee4a39d546 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1264 microversion: 2.1 time: 0.167672 Jun 10 13:43:49.643714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1613/2527] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:49 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1264 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:50.660355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f2cb2047-e5c6-460d-9416-07739578ead8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:50.668581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f2cb2047-e5c6-460d-9416-07739578ead8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:50.670423 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f2cb2047-e5c6-460d-9416-07739578ead8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:50.673567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f2cb2047-e5c6-460d-9416-07739578ead8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:50.674372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f2cb2047-e5c6-460d-9416-07739578ead8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:50.674813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f2cb2047-e5c6-460d-9416-07739578ead8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:50.829106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f2cb2047-e5c6-460d-9416-07739578ead8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:50.846916 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f2cb2047-e5c6-460d-9416-07739578ead8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1264 microversion: 2.1 time: 0.190448 Jun 10 13:43:50.856575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1614/2528] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:50 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1264 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:43:51.881173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e0f385e4-3391-43bb-90e4-7795d646782c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:51.881841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e0f385e4-3391-43bb-90e4-7795d646782c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:51.891027 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0f385e4-3391-43bb-90e4-7795d646782c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:51.891767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0f385e4-3391-43bb-90e4-7795d646782c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:51.895090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0f385e4-3391-43bb-90e4-7795d646782c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:51.896537 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e0f385e4-3391-43bb-90e4-7795d646782c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:51.943410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:43:52.215166 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e0f385e4-3391-43bb-90e4-7795d646782c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:52.223981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e0f385e4-3391-43bb-90e4-7795d646782c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1264 microversion: 2.1 time: 0.363742 Jun 10 13:43:52.225447 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1615/2529] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:51 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1264 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:53.242767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-939af796-f186-40a2-8c5f-83f56efd5b6f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:53.243493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-939af796-f186-40a2-8c5f-83f56efd5b6f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:53.246636 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:43:53.254017 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-939af796-f186-40a2-8c5f-83f56efd5b6f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:53.254471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-939af796-f186-40a2-8c5f-83f56efd5b6f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:53.255360 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-939af796-f186-40a2-8c5f-83f56efd5b6f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:53.256150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-939af796-f186-40a2-8c5f-83f56efd5b6f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:53.400609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-939af796-f186-40a2-8c5f-83f56efd5b6f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:53.409290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-939af796-f186-40a2-8c5f-83f56efd5b6f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1264 microversion: 2.1 time: 0.171297 Jun 10 13:43:53.418840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1616/2530] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:53 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1264 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:54.426339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bbf14427-ff8b-4082-a58a-b46affde1eb2 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:54.426950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bbf14427-ff8b-4082-a58a-b46affde1eb2 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:54.429022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=8 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:43:54.434397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bbf14427-ff8b-4082-a58a-b46affde1eb2 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:54.435121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bbf14427-ff8b-4082-a58a-b46affde1eb2 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:54.436087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bbf14427-ff8b-4082-a58a-b46affde1eb2 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:54.436530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bbf14427-ff8b-4082-a58a-b46affde1eb2 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:54.584014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bbf14427-ff8b-4082-a58a-b46affde1eb2 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:54.592947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bbf14427-ff8b-4082-a58a-b46affde1eb2 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1264 microversion: 2.1 time: 0.170151 Jun 10 13:43:54.604975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1617/2531] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:54 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1264 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:55.612342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5ab90e44-d3e7-404b-9b7b-eec5c0029372 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:55.613211 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5ab90e44-d3e7-404b-9b7b-eec5c0029372 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:55.619940 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5ab90e44-d3e7-404b-9b7b-eec5c0029372 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:55.620458 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5ab90e44-d3e7-404b-9b7b-eec5c0029372 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:55.621318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5ab90e44-d3e7-404b-9b7b-eec5c0029372 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:55.621762 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5ab90e44-d3e7-404b-9b7b-eec5c0029372 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:55.782126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5ab90e44-d3e7-404b-9b7b-eec5c0029372 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:55.787796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5ab90e44-d3e7-404b-9b7b-eec5c0029372 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1264 microversion: 2.1 time: 0.179601 Jun 10 13:43:55.794673 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1618/2532] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:55 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1264 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:56.804947 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-da6c5d1f-7d43-4b07-a5cb-8cb56569dec8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:56.806224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-da6c5d1f-7d43-4b07-a5cb-8cb56569dec8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:56.815779 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-da6c5d1f-7d43-4b07-a5cb-8cb56569dec8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:56.816244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-da6c5d1f-7d43-4b07-a5cb-8cb56569dec8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:56.817054 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-da6c5d1f-7d43-4b07-a5cb-8cb56569dec8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:56.817499 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-da6c5d1f-7d43-4b07-a5cb-8cb56569dec8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:57.008097 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-da6c5d1f-7d43-4b07-a5cb-8cb56569dec8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:57.022318 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-da6c5d1f-7d43-4b07-a5cb-8cb56569dec8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1264 microversion: 2.1 time: 0.221210 Jun 10 13:43:57.037929 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1619/2533] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:56 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1264 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:43:58.043910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-09e2eefa-98e1-4fe9-8b18-0f525b81775f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:58.046075 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-09e2eefa-98e1-4fe9-8b18-0f525b81775f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:58.054571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-09e2eefa-98e1-4fe9-8b18-0f525b81775f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:58.054992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-09e2eefa-98e1-4fe9-8b18-0f525b81775f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:58.056297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-09e2eefa-98e1-4fe9-8b18-0f525b81775f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:58.056944 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-09e2eefa-98e1-4fe9-8b18-0f525b81775f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:58.242109 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-09e2eefa-98e1-4fe9-8b18-0f525b81775f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:58.251905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-09e2eefa-98e1-4fe9-8b18-0f525b81775f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1264 microversion: 2.1 time: 0.211808 Jun 10 13:43:58.252476 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1620/2534] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:58 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1264 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:43:59.268680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-899fa454-152e-46d0-8c88-00f11e5e3d2b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:43:59.270019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-899fa454-152e-46d0-8c88-00f11e5e3d2b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:43:59.291515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-899fa454-152e-46d0-8c88-00f11e5e3d2b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:59.292198 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-899fa454-152e-46d0-8c88-00f11e5e3d2b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:59.293471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-899fa454-152e-46d0-8c88-00f11e5e3d2b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:43:59.294235 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-899fa454-152e-46d0-8c88-00f11e5e3d2b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:43:59.515792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-899fa454-152e-46d0-8c88-00f11e5e3d2b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:43:59.524826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-899fa454-152e-46d0-8c88-00f11e5e3d2b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1264 microversion: 2.1 time: 0.260954 Jun 10 13:43:59.525639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1621/2535] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:43:59 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1264 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:00.042691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-b99205d1-7823-486e-8369-f67919b44680 req-d5609eca-0bf3-42bc-a67f-1d1b044395a1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "53f34c67-563c-4a95-ac23-25e7d05d305d", "name": "network-vif-plugged", "status": "completed", "tag": "c60dc47c-8f9b-48bf-b642-4c2f82a01fe6"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:44:00.053236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-b99205d1-7823-486e-8369-f67919b44680 req-d5609eca-0bf3-42bc-a67f-1d1b044395a1 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:00.053767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-b99205d1-7823-486e-8369-f67919b44680 req-d5609eca-0bf3-42bc-a67f-1d1b044395a1 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:00.074103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-b99205d1-7823-486e-8369-f67919b44680 req-d5609eca-0bf3-42bc-a67f-1d1b044395a1 service nova] Creating event network-vif-plugged:c60dc47c-8f9b-48bf-b642-4c2f82a01fe6 for instance 53f34c67-563c-4a95-ac23-25e7d05d305d on hv-cinder-maste Jun 10 13:44:00.080622 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-b99205d1-7823-486e-8369-f67919b44680 req-d5609eca-0bf3-42bc-a67f-1d1b044395a1 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041518 Jun 10 13:44:00.084236 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1622/2536] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:44:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:44:00.547848 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d5eba6ac-42d2-48bc-a925-b96bcd8ece2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:00.548621 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d5eba6ac-42d2-48bc-a925-b96bcd8ece2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:00.555853 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5eba6ac-42d2-48bc-a925-b96bcd8ece2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:00.560922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5eba6ac-42d2-48bc-a925-b96bcd8ece2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:00.560922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5eba6ac-42d2-48bc-a925-b96bcd8ece2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:00.560922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d5eba6ac-42d2-48bc-a925-b96bcd8ece2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:00.722882 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d5eba6ac-42d2-48bc-a925-b96bcd8ece2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:00.731183 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d5eba6ac-42d2-48bc-a925-b96bcd8ece2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1264 microversion: 2.1 time: 0.192638 Jun 10 13:44:00.731781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1623/2537] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:00 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1264 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:01.749473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-949fa19d-4f3a-4f9d-ac2e-600fe3b29acd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:01.750133 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-949fa19d-4f3a-4f9d-ac2e-600fe3b29acd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:01.758072 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-949fa19d-4f3a-4f9d-ac2e-600fe3b29acd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:01.758522 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-949fa19d-4f3a-4f9d-ac2e-600fe3b29acd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:01.759290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-949fa19d-4f3a-4f9d-ac2e-600fe3b29acd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:01.759726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-949fa19d-4f3a-4f9d-ac2e-600fe3b29acd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:01.898768 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-949fa19d-4f3a-4f9d-ac2e-600fe3b29acd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:01.901991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-949fa19d-4f3a-4f9d-ac2e-600fe3b29acd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1264 microversion: 2.1 time: 0.157112 Jun 10 13:44:01.902651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1624/2538] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:01 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1264 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:02.919122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-09b10cb3-601c-4083-aefa-e7f334d415c1 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:02.920446 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-09b10cb3-601c-4083-aefa-e7f334d415c1 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:02.927294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-09b10cb3-601c-4083-aefa-e7f334d415c1 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:02.927905 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-09b10cb3-601c-4083-aefa-e7f334d415c1 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:02.928852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-09b10cb3-601c-4083-aefa-e7f334d415c1 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:02.929287 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-09b10cb3-601c-4083-aefa-e7f334d415c1 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:03.085981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-09b10cb3-601c-4083-aefa-e7f334d415c1 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:03.094831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-09b10cb3-601c-4083-aefa-e7f334d415c1 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1450 microversion: 2.1 time: 0.179450 Jun 10 13:44:03.105575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1625/2539] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:02 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1450 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:03.114207 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-32fd7762-0ff7-4212-bf8f-2cbb68cac4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:03.114896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-32fd7762-0ff7-4212-bf8f-2cbb68cac4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:03.124600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-32fd7762-0ff7-4212-bf8f-2cbb68cac4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:03.124600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-32fd7762-0ff7-4212-bf8f-2cbb68cac4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:03.124600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-32fd7762-0ff7-4212-bf8f-2cbb68cac4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:03.124600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-32fd7762-0ff7-4212-bf8f-2cbb68cac4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:03.289640 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-32fd7762-0ff7-4212-bf8f-2cbb68cac4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:03.298335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-32fd7762-0ff7-4212-bf8f-2cbb68cac4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1450 microversion: 2.1 time: 0.190441 Jun 10 13:44:03.299223 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1626/2540] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:03 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1450 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:44:05.846944 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-52ad58aa-f0c9-4195-941d-77fe1437cd1a req-ae0aa99f-a7cb-4bb7-9bc7-8e8b4595f008 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "53f34c67-563c-4a95-ac23-25e7d05d305d", "tag": "c60dc47c-8f9b-48bf-b642-4c2f82a01fe6"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:44:05.855877 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-52ad58aa-f0c9-4195-941d-77fe1437cd1a req-ae0aa99f-a7cb-4bb7-9bc7-8e8b4595f008 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:05.856324 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-52ad58aa-f0c9-4195-941d-77fe1437cd1a req-ae0aa99f-a7cb-4bb7-9bc7-8e8b4595f008 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:05.876295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-52ad58aa-f0c9-4195-941d-77fe1437cd1a req-ae0aa99f-a7cb-4bb7-9bc7-8e8b4595f008 service nova] Creating event network-changed:c60dc47c-8f9b-48bf-b642-4c2f82a01fe6 for instance 53f34c67-563c-4a95-ac23-25e7d05d305d on hv-cinder-maste Jun 10 13:44:05.882103 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-52ad58aa-f0c9-4195-941d-77fe1437cd1a req-ae0aa99f-a7cb-4bb7-9bc7-8e8b4595f008 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038782 Jun 10 13:44:05.882789 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1627/2541] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:44:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:44:06.309367 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7df0b8c1-5970-4d26-939e-8755cf95a0c1 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Action: 'create', calling method: .version_select at 0x7f9305411280>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1741328647", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMb+C9jD6Ojni3fmZfqcjoyoFZsD/7n8/TDm/GwTk28ohTIb/Zo9R1nqO8oNd3nZf/u6b+9abAS/48YDDw4iX2A5Fo02pIy/QgRE7RYETxcbq0awO4+7UY0vwRUFM2MPlQ=="}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:44:06.314521 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=83,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:44:06.324975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-7df0b8c1-5970-4d26-939e-8755cf95a0c1 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Getting quotas for project ea8e34865f774cdf830b60908416ee9a. Resources: {'key_pairs'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:44:06.329617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-7df0b8c1-5970-4d26-939e-8755cf95a0c1 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Getting quotas for user ad44d340bc1d4cf48694416948333102 and project ea8e34865f774cdf830b60908416ee9a. Resources: {'key_pairs'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:44:06.410350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7df0b8c1-5970-4d26-939e-8755cf95a0c1 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.105378 Jun 10 13:44:06.421787 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1628/2542] 10.222.0.25 () {60 vars in 1158 bytes} [Fri Jun 10 13:44:06 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:44:06.462995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1411040348", "imageRef": "da80ab79-d5fc-4940-9380-c3105bbb4659", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1741328647", "networks": [{"uuid": "da6a8444-4861-4ebe-999e-e0ad24280d91"}]}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:44:06.514087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.network.neutron [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] validate_networks() for [('da6a8444-4861-4ebe-999e-e0ad24280d91', None, None, None, None, None)] {{(pid=118092) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:44:06.885409 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Flavor limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:44:06.885970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Image limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:44:06.886478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Flavor pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:44:06.886956 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Image pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:44:06.887442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:44:06.897134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:06.897591 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:06.898392 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:06.898999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:06.915791 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Getting quotas for project ea8e34865f774cdf830b60908416ee9a. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:44:06.920930 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Getting quotas for user ad44d340bc1d4cf48694416948333102 and project ea8e34865f774cdf830b60908416ee9a. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:44:06.931441 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Going to run 1 instances... {{(pid=118092) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:44:06.933954 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='da80ab79-d5fc-4940-9380-c3105bbb4659',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118092) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:44:07.128201 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ed353060-57b5-4219-942c-d40b1682105d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.670185 Jun 10 13:44:07.128866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1629/2543] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:44:06 2022] POST /compute/v2.1/servers => generated 380 bytes in 672 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jun 10 13:44:07.138295 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-237c4966-451f-4bea-91de-b48464bcd612 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:07.139353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-237c4966-451f-4bea-91de-b48464bcd612 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:07.194119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-237c4966-451f-4bea-91de-b48464bcd612 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:07.200712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-237c4966-451f-4bea-91de-b48464bcd612 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1259 microversion: 2.1 time: 0.066090 Jun 10 13:44:07.202015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1630/2544] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:07 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1259 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:08.218667 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-246d4859-5711-4f01-9555-00ad94d9c3d7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:08.219269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-246d4859-5711-4f01-9555-00ad94d9c3d7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:08.226482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-246d4859-5711-4f01-9555-00ad94d9c3d7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:08.226932 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-246d4859-5711-4f01-9555-00ad94d9c3d7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:08.227676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-246d4859-5711-4f01-9555-00ad94d9c3d7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:08.228104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-246d4859-5711-4f01-9555-00ad94d9c3d7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:08.290284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-246d4859-5711-4f01-9555-00ad94d9c3d7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:08.302826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-246d4859-5711-4f01-9555-00ad94d9c3d7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1263 microversion: 2.1 time: 0.088119 Jun 10 13:44:08.304440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1631/2545] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:08 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1263 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:09.320367 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-85eedb88-5452-4bb6-a47c-8b574e32c168 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:09.321045 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-85eedb88-5452-4bb6-a47c-8b574e32c168 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:09.327696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-85eedb88-5452-4bb6-a47c-8b574e32c168 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:09.328145 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-85eedb88-5452-4bb6-a47c-8b574e32c168 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:09.328940 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-85eedb88-5452-4bb6-a47c-8b574e32c168 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:09.329381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-85eedb88-5452-4bb6-a47c-8b574e32c168 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:09.412191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-85eedb88-5452-4bb6-a47c-8b574e32c168 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:09.420952 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-85eedb88-5452-4bb6-a47c-8b574e32c168 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1319 microversion: 2.1 time: 0.104661 Jun 10 13:44:09.421828 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1632/2546] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:09 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1319 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:10.438989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-6cbc3399-6ee4-458b-951d-14326e0639ff tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:10.439924 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-6cbc3399-6ee4-458b-951d-14326e0639ff tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:10.450041 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6cbc3399-6ee4-458b-951d-14326e0639ff tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:10.450483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6cbc3399-6ee4-458b-951d-14326e0639ff tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:10.451248 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6cbc3399-6ee4-458b-951d-14326e0639ff tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:10.452922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-6cbc3399-6ee4-458b-951d-14326e0639ff tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:10.638160 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-6cbc3399-6ee4-458b-951d-14326e0639ff tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:10.649063 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-6cbc3399-6ee4-458b-951d-14326e0639ff tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.214532 Jun 10 13:44:10.663120 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1633/2547] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:10 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:44:11.670795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-45c2200d-d39d-4244-8eb5-3fb676b5311d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:11.671995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-45c2200d-d39d-4244-8eb5-3fb676b5311d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:11.682617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45c2200d-d39d-4244-8eb5-3fb676b5311d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:11.683293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45c2200d-d39d-4244-8eb5-3fb676b5311d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:11.684071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45c2200d-d39d-4244-8eb5-3fb676b5311d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:11.684503 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-45c2200d-d39d-4244-8eb5-3fb676b5311d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:11.824159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-45c2200d-d39d-4244-8eb5-3fb676b5311d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:11.832285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-45c2200d-d39d-4244-8eb5-3fb676b5311d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.165868 Jun 10 13:44:11.841571 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1634/2548] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:11 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:12.849455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a476e652-b8f3-4acd-9a03-f02a16c9afaa tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:12.850119 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a476e652-b8f3-4acd-9a03-f02a16c9afaa tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:12.856581 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a476e652-b8f3-4acd-9a03-f02a16c9afaa tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:12.857050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a476e652-b8f3-4acd-9a03-f02a16c9afaa tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:12.857805 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a476e652-b8f3-4acd-9a03-f02a16c9afaa tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:12.858931 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a476e652-b8f3-4acd-9a03-f02a16c9afaa tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:12.995904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-a476e652-b8f3-4acd-9a03-f02a16c9afaa tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:13.003872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a476e652-b8f3-4acd-9a03-f02a16c9afaa tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.158154 Jun 10 13:44:13.004566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1635/2549] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:12 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:14.020902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b5c911d4-a753-4165-b521-ddf9d4073bc7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:14.021629 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b5c911d4-a753-4165-b521-ddf9d4073bc7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:14.028263 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5c911d4-a753-4165-b521-ddf9d4073bc7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:14.028738 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5c911d4-a753-4165-b521-ddf9d4073bc7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:14.029534 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5c911d4-a753-4165-b521-ddf9d4073bc7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:14.030546 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b5c911d4-a753-4165-b521-ddf9d4073bc7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:14.034540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:44:14.167619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b5c911d4-a753-4165-b521-ddf9d4073bc7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:14.175895 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b5c911d4-a753-4165-b521-ddf9d4073bc7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.158833 Jun 10 13:44:14.177529 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1636/2550] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:14 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:15.195391 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-78578a22-d9e1-4069-93b4-4652a98470f2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:15.196863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-78578a22-d9e1-4069-93b4-4652a98470f2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:15.203964 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78578a22-d9e1-4069-93b4-4652a98470f2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:15.204589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78578a22-d9e1-4069-93b4-4652a98470f2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:15.205398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78578a22-d9e1-4069-93b4-4652a98470f2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:15.205833 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-78578a22-d9e1-4069-93b4-4652a98470f2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:15.361055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-78578a22-d9e1-4069-93b4-4652a98470f2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:15.369361 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-78578a22-d9e1-4069-93b4-4652a98470f2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.178375 Jun 10 13:44:15.370344 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1637/2551] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:15 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:16.387259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f94df144-15ce-4dfc-b896-bde4488d020c tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:16.387950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f94df144-15ce-4dfc-b896-bde4488d020c tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:16.394611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f94df144-15ce-4dfc-b896-bde4488d020c tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:16.395059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f94df144-15ce-4dfc-b896-bde4488d020c tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:16.395874 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f94df144-15ce-4dfc-b896-bde4488d020c tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:16.396306 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f94df144-15ce-4dfc-b896-bde4488d020c tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:16.541872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f94df144-15ce-4dfc-b896-bde4488d020c tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:16.550094 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f94df144-15ce-4dfc-b896-bde4488d020c tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.166776 Jun 10 13:44:16.550893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1638/2552] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:16 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:17.566799 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-95a14222-e269-4c80-8f3d-8568e20c5558 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:17.567436 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-95a14222-e269-4c80-8f3d-8568e20c5558 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:17.569596 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:44:17.570299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=12 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:44:17.578069 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95a14222-e269-4c80-8f3d-8568e20c5558 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:17.578518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95a14222-e269-4c80-8f3d-8568e20c5558 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:17.579289 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95a14222-e269-4c80-8f3d-8568e20c5558 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:17.579726 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-95a14222-e269-4c80-8f3d-8568e20c5558 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:17.739233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-95a14222-e269-4c80-8f3d-8568e20c5558 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:17.746922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-95a14222-e269-4c80-8f3d-8568e20c5558 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.183896 Jun 10 13:44:17.747514 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1639/2553] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:17 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:18.763753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-648a0d8f-5af8-4ba0-9e7a-2d0593b8db20 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:18.764479 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-648a0d8f-5af8-4ba0-9e7a-2d0593b8db20 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:18.770810 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-648a0d8f-5af8-4ba0-9e7a-2d0593b8db20 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:18.771253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-648a0d8f-5af8-4ba0-9e7a-2d0593b8db20 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:18.771998 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-648a0d8f-5af8-4ba0-9e7a-2d0593b8db20 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:18.772435 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-648a0d8f-5af8-4ba0-9e7a-2d0593b8db20 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:18.902535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-648a0d8f-5af8-4ba0-9e7a-2d0593b8db20 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:18.910705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-648a0d8f-5af8-4ba0-9e7a-2d0593b8db20 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.150879 Jun 10 13:44:18.911266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1640/2554] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:18 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:19.927539 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c31555d7-289a-4a29-9803-0d955ec3eac7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:19.928152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c31555d7-289a-4a29-9803-0d955ec3eac7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:19.934179 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c31555d7-289a-4a29-9803-0d955ec3eac7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:19.934632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c31555d7-289a-4a29-9803-0d955ec3eac7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:19.935390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c31555d7-289a-4a29-9803-0d955ec3eac7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:19.935822 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c31555d7-289a-4a29-9803-0d955ec3eac7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:20.059781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c31555d7-289a-4a29-9803-0d955ec3eac7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:20.067827 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c31555d7-289a-4a29-9803-0d955ec3eac7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.144351 Jun 10 13:44:20.068394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1641/2555] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:19 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:21.080437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-9521bc0a-565e-4234-a9d0-3e083f0a3db7 req-0469a617-3945-4d32-8c4e-340b4452a847 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "80cb1520-6b76-4212-82d5-b5ba482b481f", "tag": "2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:44:21.091191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-9521bc0a-565e-4234-a9d0-3e083f0a3db7 req-0469a617-3945-4d32-8c4e-340b4452a847 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:21.091721 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-9521bc0a-565e-4234-a9d0-3e083f0a3db7 req-0469a617-3945-4d32-8c4e-340b4452a847 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:21.104008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-9521bc0a-565e-4234-a9d0-3e083f0a3db7 req-0469a617-3945-4d32-8c4e-340b4452a847 service nova] Creating event network-changed:2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a for instance 80cb1520-6b76-4212-82d5-b5ba482b481f on hv-cinder-maste Jun 10 13:44:21.109333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-9521bc0a-565e-4234-a9d0-3e083f0a3db7 req-0469a617-3945-4d32-8c4e-340b4452a847 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034979 Jun 10 13:44:21.116006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1642/2556] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:44:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:44:21.116006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f0caf56f-c7ea-4128-8186-795d55d9a620 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:21.116006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f0caf56f-c7ea-4128-8186-795d55d9a620 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:21.122426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0caf56f-c7ea-4128-8186-795d55d9a620 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:21.122878 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0caf56f-c7ea-4128-8186-795d55d9a620 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:21.123651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0caf56f-c7ea-4128-8186-795d55d9a620 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:21.124686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f0caf56f-c7ea-4128-8186-795d55d9a620 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:21.328165 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f0caf56f-c7ea-4128-8186-795d55d9a620 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:21.336035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f0caf56f-c7ea-4128-8186-795d55d9a620 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.224812 Jun 10 13:44:21.337077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1643/2557] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:21 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:44:22.352978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-26d9be33-4f5d-4e20-8e92-f6ca483e648b tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:22.353596 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-26d9be33-4f5d-4e20-8e92-f6ca483e648b tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:22.360124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26d9be33-4f5d-4e20-8e92-f6ca483e648b tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:22.360576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26d9be33-4f5d-4e20-8e92-f6ca483e648b tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:22.361381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26d9be33-4f5d-4e20-8e92-f6ca483e648b tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:22.361824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-26d9be33-4f5d-4e20-8e92-f6ca483e648b tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:22.712655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-26d9be33-4f5d-4e20-8e92-f6ca483e648b tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:22.720679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-26d9be33-4f5d-4e20-8e92-f6ca483e648b tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.371575 Jun 10 13:44:22.721273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1644/2558] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:22 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:23.737403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-31bc137e-6702-482f-b39b-bfd3b2b6b87e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:23.738014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-31bc137e-6702-482f-b39b-bfd3b2b6b87e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:23.744540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31bc137e-6702-482f-b39b-bfd3b2b6b87e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:23.745024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31bc137e-6702-482f-b39b-bfd3b2b6b87e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:23.745809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31bc137e-6702-482f-b39b-bfd3b2b6b87e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:23.746247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31bc137e-6702-482f-b39b-bfd3b2b6b87e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:23.868512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-31bc137e-6702-482f-b39b-bfd3b2b6b87e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:23.876381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-31bc137e-6702-482f-b39b-bfd3b2b6b87e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.142802 Jun 10 13:44:23.877008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1645/2559] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:23 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:24.893862 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-936ed958-33f3-426d-9cbe-236df7a2d2b0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:24.894498 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-936ed958-33f3-426d-9cbe-236df7a2d2b0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:24.900924 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-936ed958-33f3-426d-9cbe-236df7a2d2b0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:24.901372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-936ed958-33f3-426d-9cbe-236df7a2d2b0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:24.902123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-936ed958-33f3-426d-9cbe-236df7a2d2b0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:24.902556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-936ed958-33f3-426d-9cbe-236df7a2d2b0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:25.029002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-936ed958-33f3-426d-9cbe-236df7a2d2b0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:25.036349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-936ed958-33f3-426d-9cbe-236df7a2d2b0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.147055 Jun 10 13:44:25.036937 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1646/2560] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:24 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:26.053092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-35b2ddfe-c029-4b38-a037-a9fd5bdf1358 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:26.053790 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-35b2ddfe-c029-4b38-a037-a9fd5bdf1358 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:26.060338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35b2ddfe-c029-4b38-a037-a9fd5bdf1358 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:26.060821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35b2ddfe-c029-4b38-a037-a9fd5bdf1358 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:26.061585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35b2ddfe-c029-4b38-a037-a9fd5bdf1358 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:26.062020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-35b2ddfe-c029-4b38-a037-a9fd5bdf1358 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:26.187143 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-35b2ddfe-c029-4b38-a037-a9fd5bdf1358 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:26.194957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-35b2ddfe-c029-4b38-a037-a9fd5bdf1358 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.145494 Jun 10 13:44:26.195517 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1647/2561] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:26 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:27.211892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3dc8d3c1-7226-4d74-9266-80c9f5de9e0f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:27.212523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3dc8d3c1-7226-4d74-9266-80c9f5de9e0f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:27.218983 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3dc8d3c1-7226-4d74-9266-80c9f5de9e0f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:27.219428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3dc8d3c1-7226-4d74-9266-80c9f5de9e0f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:27.220182 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3dc8d3c1-7226-4d74-9266-80c9f5de9e0f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:27.220613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3dc8d3c1-7226-4d74-9266-80c9f5de9e0f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:27.356233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3dc8d3c1-7226-4d74-9266-80c9f5de9e0f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:27.364331 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3dc8d3c1-7226-4d74-9266-80c9f5de9e0f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.156318 Jun 10 13:44:27.364922 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1648/2562] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:27 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:28.066297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-35fe575d-3425-482a-9ccf-879080011432 req-19103851-b3ee-47a5-931a-d9e857e931c0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "80cb1520-6b76-4212-82d5-b5ba482b481f", "name": "network-vif-plugged", "status": "completed", "tag": "2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:44:28.074867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-35fe575d-3425-482a-9ccf-879080011432 req-19103851-b3ee-47a5-931a-d9e857e931c0 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:28.075313 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-35fe575d-3425-482a-9ccf-879080011432 req-19103851-b3ee-47a5-931a-d9e857e931c0 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:28.086769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-35fe575d-3425-482a-9ccf-879080011432 req-19103851-b3ee-47a5-931a-d9e857e931c0 service nova] Creating event network-vif-plugged:2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a for instance 80cb1520-6b76-4212-82d5-b5ba482b481f on hv-cinder-maste Jun 10 13:44:28.091812 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-35fe575d-3425-482a-9ccf-879080011432 req-19103851-b3ee-47a5-931a-d9e857e931c0 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029385 Jun 10 13:44:28.092442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1649/2563] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:44:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:44:28.382856 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1c3b8a2d-fbf1-4257-86d3-9ca428fd1060 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:28.383556 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1c3b8a2d-fbf1-4257-86d3-9ca428fd1060 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:28.389868 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c3b8a2d-fbf1-4257-86d3-9ca428fd1060 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:28.390260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c3b8a2d-fbf1-4257-86d3-9ca428fd1060 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:28.391022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c3b8a2d-fbf1-4257-86d3-9ca428fd1060 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:28.392044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c3b8a2d-fbf1-4257-86d3-9ca428fd1060 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:28.542570 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1c3b8a2d-fbf1-4257-86d3-9ca428fd1060 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:28.550243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1c3b8a2d-fbf1-4257-86d3-9ca428fd1060 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.172087 Jun 10 13:44:28.550823 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1650/2564] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:28 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:29.567748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5107abba-8781-47ac-bc5d-2196a2fa434a tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:29.568451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5107abba-8781-47ac-bc5d-2196a2fa434a tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:29.575260 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5107abba-8781-47ac-bc5d-2196a2fa434a tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:29.575716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5107abba-8781-47ac-bc5d-2196a2fa434a tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:29.576506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5107abba-8781-47ac-bc5d-2196a2fa434a tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:29.576980 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5107abba-8781-47ac-bc5d-2196a2fa434a tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:29.701175 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5107abba-8781-47ac-bc5d-2196a2fa434a tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:29.709202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5107abba-8781-47ac-bc5d-2196a2fa434a tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1359 microversion: 2.1 time: 0.145950 Jun 10 13:44:29.709768 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1651/2565] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:29 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1359 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:30.726494 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-07d6026a-a358-45f6-a88f-5605256e4ac2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:30.727106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-07d6026a-a358-45f6-a88f-5605256e4ac2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:30.733590 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07d6026a-a358-45f6-a88f-5605256e4ac2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:30.734039 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07d6026a-a358-45f6-a88f-5605256e4ac2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:30.734797 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07d6026a-a358-45f6-a88f-5605256e4ac2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:30.735880 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-07d6026a-a358-45f6-a88f-5605256e4ac2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:30.859908 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-07d6026a-a358-45f6-a88f-5605256e4ac2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:30.867954 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-07d6026a-a358-45f6-a88f-5605256e4ac2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1549 microversion: 2.1 time: 0.145278 Jun 10 13:44:30.868516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1652/2566] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:30 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1549 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:30.887739 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-bac67ece-a32e-4d17-94a7-4b38f9855648 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:30.888342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-bac67ece-a32e-4d17-94a7-4b38f9855648 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:30.894755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bac67ece-a32e-4d17-94a7-4b38f9855648 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:30.895197 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bac67ece-a32e-4d17-94a7-4b38f9855648 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:30.895945 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bac67ece-a32e-4d17-94a7-4b38f9855648 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:30.896370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-bac67ece-a32e-4d17-94a7-4b38f9855648 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:31.024031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-bac67ece-a32e-4d17-94a7-4b38f9855648 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:31.031412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-bac67ece-a32e-4d17-94a7-4b38f9855648 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1549 microversion: 2.1 time: 0.147338 Jun 10 13:44:31.031972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1653/2567] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:30 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1549 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:31.046891 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f03b6071-1600-44e9-81f9-addb3a6fe1f7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:31.050275 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f03b6071-1600-44e9-81f9-addb3a6fe1f7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:31.051025 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f03b6071-1600-44e9-81f9-addb3a6fe1f7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Searching by: {'deleted': False, 'project_id': 'ea8e34865f774cdf830b60908416ee9a'} {{(pid=118092) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jun 10 13:44:31.057274 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f03b6071-1600-44e9-81f9-addb3a6fe1f7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:31.057715 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f03b6071-1600-44e9-81f9-addb3a6fe1f7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:31.058495 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f03b6071-1600-44e9-81f9-addb3a6fe1f7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:31.058919 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f03b6071-1600-44e9-81f9-addb3a6fe1f7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:31.125415 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=100 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:44:31.144731 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.multi_cell_list [None req-f03b6071-1600-44e9-81f9-addb3a6fe1f7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=118092) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jun 10 13:44:31.233836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f03b6071-1600-44e9-81f9-addb3a6fe1f7 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.190161 Jun 10 13:44:31.234338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1654/2568] 10.222.0.25 () {58 vars in 1124 bytes} [Fri Jun 10 13:44:31 2022] GET /compute/v2.1/servers => generated 343 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:44:31.243895 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-31cd9397-e5d7-446d-af35-2e554a0e9238 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:31.244491 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-31cd9397-e5d7-446d-af35-2e554a0e9238 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:31.250174 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31cd9397-e5d7-446d-af35-2e554a0e9238 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:31.250622 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31cd9397-e5d7-446d-af35-2e554a0e9238 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:31.251902 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31cd9397-e5d7-446d-af35-2e554a0e9238 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:31.252329 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-31cd9397-e5d7-446d-af35-2e554a0e9238 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:31.380671 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-31cd9397-e5d7-446d-af35-2e554a0e9238 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:31.388073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-31cd9397-e5d7-446d-af35-2e554a0e9238 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1549 microversion: 2.1 time: 0.147610 Jun 10 13:44:31.388631 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1655/2569] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:31 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1549 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:33.243595 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0ea4bccd-927c-4eed-bd83-d6cc34fad0f8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d8298772-9338-4b96-b09c-db39371e8088"}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:44:33.244645 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0ea4bccd-927c-4eed-bd83-d6cc34fad0f8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:33.251742 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ea4bccd-927c-4eed-bd83-d6cc34fad0f8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:33.256668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ea4bccd-927c-4eed-bd83-d6cc34fad0f8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:33.256668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ea4bccd-927c-4eed-bd83-d6cc34fad0f8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:33.256668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ea4bccd-927c-4eed-bd83-d6cc34fad0f8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:33.914266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0ea4bccd-927c-4eed-bd83-d6cc34fad0f8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "POST /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.674306 Jun 10 13:44:33.914940 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1656/2570] 10.222.0.25 () {60 vars in 1322 bytes} [Fri Jun 10 13:44:33 2022] POST /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments => generated 194 bytes in 676 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:44:41.439418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:44:41.443540 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-53b1cdf8-8848-4c52-8937-23ca9795f3e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:41.444231 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-53b1cdf8-8848-4c52-8937-23ca9795f3e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:41.446322 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=82,nova_api:INSERT=4 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:44:41.452481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53b1cdf8-8848-4c52-8937-23ca9795f3e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:41.452481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53b1cdf8-8848-4c52-8937-23ca9795f3e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:41.452481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53b1cdf8-8848-4c52-8937-23ca9795f3e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:41.452830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-53b1cdf8-8848-4c52-8937-23ca9795f3e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:41.583205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-53b1cdf8-8848-4c52-8937-23ca9795f3e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:41.590973 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-53b1cdf8-8848-4c52-8937-23ca9795f3e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1595 microversion: 2.1 time: 0.153147 Jun 10 13:44:41.591535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1657/2571] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:41 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1595 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:44:43.682965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-1c5c6430-3c06-415e-947d-988e22365598 req-30e6b933-f1e1-48fc-a53a-524f0aeceb2f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "80cb1520-6b76-4212-82d5-b5ba482b481f", "tag": "2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:44:43.691839 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-1c5c6430-3c06-415e-947d-988e22365598 req-30e6b933-f1e1-48fc-a53a-524f0aeceb2f service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:43.692566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-1c5c6430-3c06-415e-947d-988e22365598 req-30e6b933-f1e1-48fc-a53a-524f0aeceb2f service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:43.704605 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-1c5c6430-3c06-415e-947d-988e22365598 req-30e6b933-f1e1-48fc-a53a-524f0aeceb2f service nova] Creating event network-changed:2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a for instance 80cb1520-6b76-4212-82d5-b5ba482b481f on hv-cinder-maste Jun 10 13:44:43.710703 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-1c5c6430-3c06-415e-947d-988e22365598 req-30e6b933-f1e1-48fc-a53a-524f0aeceb2f service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031189 Jun 10 13:44:43.711599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1658/2572] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:44:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:44:46.005954 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-4c75370f-6cb0-4289-a9e3-e540d682edf2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-2085704531"}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:44:46.006598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-4c75370f-6cb0-4289-a9e3-e540d682edf2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:46.012843 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4c75370f-6cb0-4289-a9e3-e540d682edf2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:46.013293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4c75370f-6cb0-4289-a9e3-e540d682edf2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:46.014046 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4c75370f-6cb0-4289-a9e3-e540d682edf2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:46.014481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-4c75370f-6cb0-4289-a9e3-e540d682edf2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:46.139234 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.network.security_group_api [None req-4c75370f-6cb0-4289-a9e3-e540d682edf2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Adding security group e39e3bb1-996e-4953-a1f1-efa4b62851ec to port 2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a Jun 10 13:44:46.521060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-4c75370f-6cb0-4289-a9e3-e540d682edf2 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "POST /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/action" status: 202 len: 0 microversion: 2.1 time: 0.519828 Jun 10 13:44:46.522347 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1659/2573] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Jun 10 13:44:45 2022] POST /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/action => generated 0 bytes in 522 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:44:46.539680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3b788e10-cfe4-4d5a-a3f1-93202dba13be tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:44:46.539680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3b788e10-cfe4-4d5a-a3f1-93202dba13be tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:44:46.542981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3b788e10-cfe4-4d5a-a3f1-93202dba13be tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:46.543444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3b788e10-cfe4-4d5a-a3f1-93202dba13be tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:46.544199 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3b788e10-cfe4-4d5a-a3f1-93202dba13be tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:46.544694 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3b788e10-cfe4-4d5a-a3f1-93202dba13be tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:46.719464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3b788e10-cfe4-4d5a-a3f1-93202dba13be tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:44:46.729785 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3b788e10-cfe4-4d5a-a3f1-93202dba13be tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1760 microversion: 2.1 time: 0.203207 Jun 10 13:44:46.731080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1660/2574] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:44:46 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1760 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:44:46.871976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-7f7a005a-99b1-448d-95bc-cd83c9d22bec req-077185e3-3c34-48e2-bc4d-efdae8710db5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "80cb1520-6b76-4212-82d5-b5ba482b481f", "tag": "2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:44:46.880352 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-7f7a005a-99b1-448d-95bc-cd83c9d22bec req-077185e3-3c34-48e2-bc4d-efdae8710db5 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:44:46.880920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-7f7a005a-99b1-448d-95bc-cd83c9d22bec req-077185e3-3c34-48e2-bc4d-efdae8710db5 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:44:46.891733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-7f7a005a-99b1-448d-95bc-cd83c9d22bec req-077185e3-3c34-48e2-bc4d-efdae8710db5 service nova] Creating event network-changed:2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a for instance 80cb1520-6b76-4212-82d5-b5ba482b481f on hv-cinder-maste Jun 10 13:44:46.897176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-7f7a005a-99b1-448d-95bc-cd83c9d22bec req-077185e3-3c34-48e2-bc4d-efdae8710db5 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.164879 Jun 10 13:44:46.897852 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1661/2575] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:44:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:45:09.114987 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=9 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:45:09.115685 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=43,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:45:09.119802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dcce1d90-4bdb-4252-ad6d-a55edcdfbf30 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:45:09.120941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dcce1d90-4bdb-4252-ad6d-a55edcdfbf30 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:09.127490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcce1d90-4bdb-4252-ad6d-a55edcdfbf30 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:09.127490 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcce1d90-4bdb-4252-ad6d-a55edcdfbf30 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:09.127992 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcce1d90-4bdb-4252-ad6d-a55edcdfbf30 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:09.128425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dcce1d90-4bdb-4252-ad6d-a55edcdfbf30 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:09.378535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dcce1d90-4bdb-4252-ad6d-a55edcdfbf30 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "POST /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/action" status: 202 len: 0 microversion: 2.1 time: 0.265434 Jun 10 13:45:09.379137 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1662/2576] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Jun 10 13:45:09 2022] POST /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/action => generated 0 bytes in 267 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:45:09.387492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8bb03b84-65f6-48a9-8c09-c9dc72afd6c6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:09.391238 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8bb03b84-65f6-48a9-8c09-c9dc72afd6c6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:09.394325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8bb03b84-65f6-48a9-8c09-c9dc72afd6c6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:09.394806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8bb03b84-65f6-48a9-8c09-c9dc72afd6c6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:09.395561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8bb03b84-65f6-48a9-8c09-c9dc72afd6c6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:09.395996 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8bb03b84-65f6-48a9-8c09-c9dc72afd6c6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:09.581759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8bb03b84-65f6-48a9-8c09-c9dc72afd6c6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:09.590466 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8bb03b84-65f6-48a9-8c09-c9dc72afd6c6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1752 microversion: 2.1 time: 0.207100 Jun 10 13:45:09.591040 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1663/2577] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:09 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1752 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:10.608349 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-441b2d24-bfb9-4f8a-b415-33797ab62ecf tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:10.608991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-441b2d24-bfb9-4f8a-b415-33797ab62ecf tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:10.616011 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-441b2d24-bfb9-4f8a-b415-33797ab62ecf tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:10.616459 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-441b2d24-bfb9-4f8a-b415-33797ab62ecf tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:10.617252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-441b2d24-bfb9-4f8a-b415-33797ab62ecf tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:10.617686 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-441b2d24-bfb9-4f8a-b415-33797ab62ecf tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:10.761073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-441b2d24-bfb9-4f8a-b415-33797ab62ecf tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:10.768923 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-441b2d24-bfb9-4f8a-b415-33797ab62ecf tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1757 microversion: 2.1 time: 0.164515 Jun 10 13:45:10.769501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1664/2578] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:10 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1757 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:11.785981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-303fd029-a772-45c4-b024-b0acae0a6f97 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:11.786588 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-303fd029-a772-45c4-b024-b0acae0a6f97 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:11.792968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-303fd029-a772-45c4-b024-b0acae0a6f97 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:11.793422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-303fd029-a772-45c4-b024-b0acae0a6f97 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:11.794176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-303fd029-a772-45c4-b024-b0acae0a6f97 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:11.794608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-303fd029-a772-45c4-b024-b0acae0a6f97 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:11.942869 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-303fd029-a772-45c4-b024-b0acae0a6f97 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:11.950925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-303fd029-a772-45c4-b024-b0acae0a6f97 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1757 microversion: 2.1 time: 0.168743 Jun 10 13:45:11.951486 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1665/2579] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:11 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1757 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:12.967710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cffbc275-af13-4e11-88a3-322c8ee0593e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:12.968360 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-cffbc275-af13-4e11-88a3-322c8ee0593e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:12.974795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cffbc275-af13-4e11-88a3-322c8ee0593e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:12.975239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cffbc275-af13-4e11-88a3-322c8ee0593e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:12.975990 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cffbc275-af13-4e11-88a3-322c8ee0593e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:12.976422 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-cffbc275-af13-4e11-88a3-322c8ee0593e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:13.113085 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-cffbc275-af13-4e11-88a3-322c8ee0593e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:13.121010 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cffbc275-af13-4e11-88a3-322c8ee0593e tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1757 microversion: 2.1 time: 0.157152 Jun 10 13:45:13.121580 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1666/2580] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:12 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1757 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:14.138813 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8f789b57-d75d-4ff3-b683-cc3b3bb1ef25 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:14.139424 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8f789b57-d75d-4ff3-b683-cc3b3bb1ef25 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:14.146390 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f789b57-d75d-4ff3-b683-cc3b3bb1ef25 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:14.146835 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f789b57-d75d-4ff3-b683-cc3b3bb1ef25 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:14.147592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f789b57-d75d-4ff3-b683-cc3b3bb1ef25 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:14.148031 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8f789b57-d75d-4ff3-b683-cc3b3bb1ef25 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:14.287604 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-8f789b57-d75d-4ff3-b683-cc3b3bb1ef25 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:14.295487 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8f789b57-d75d-4ff3-b683-cc3b3bb1ef25 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1757 microversion: 2.1 time: 0.160626 Jun 10 13:45:14.296058 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1667/2581] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:14 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1757 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:15.312898 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7029069f-51ea-49b7-a940-f94cda812b61 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:15.313720 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-7029069f-51ea-49b7-a940-f94cda812b61 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:15.325912 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7029069f-51ea-49b7-a940-f94cda812b61 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:15.326455 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7029069f-51ea-49b7-a940-f94cda812b61 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:15.327296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7029069f-51ea-49b7-a940-f94cda812b61 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:15.327800 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-7029069f-51ea-49b7-a940-f94cda812b61 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:15.473083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-7029069f-51ea-49b7-a940-f94cda812b61 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:15.481337 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7029069f-51ea-49b7-a940-f94cda812b61 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1757 microversion: 2.1 time: 0.172315 Jun 10 13:45:15.481901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1668/2582] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:15 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1757 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:16.498298 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-17857933-3ab1-4fd7-960d-d419963972a5 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:16.498885 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-17857933-3ab1-4fd7-960d-d419963972a5 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:16.505833 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17857933-3ab1-4fd7-960d-d419963972a5 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:16.506281 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17857933-3ab1-4fd7-960d-d419963972a5 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:16.507034 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17857933-3ab1-4fd7-960d-d419963972a5 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:16.507465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-17857933-3ab1-4fd7-960d-d419963972a5 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:16.649792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-17857933-3ab1-4fd7-960d-d419963972a5 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:16.658046 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-17857933-3ab1-4fd7-960d-d419963972a5 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1757 microversion: 2.1 time: 0.163583 Jun 10 13:45:16.658619 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1669/2583] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:16 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1757 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:17.675688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f1e5f611-535f-46b0-a4e2-14d13365203f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:17.676283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f1e5f611-535f-46b0-a4e2-14d13365203f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:17.683269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1e5f611-535f-46b0-a4e2-14d13365203f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:17.683718 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1e5f611-535f-46b0-a4e2-14d13365203f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:17.684477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1e5f611-535f-46b0-a4e2-14d13365203f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:17.684946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f1e5f611-535f-46b0-a4e2-14d13365203f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:17.827607 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f1e5f611-535f-46b0-a4e2-14d13365203f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:17.836035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f1e5f611-535f-46b0-a4e2-14d13365203f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1757 microversion: 2.1 time: 0.164367 Jun 10 13:45:17.836599 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1670/2584] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:17 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1757 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:45:18.853413 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-953fd14f-b8eb-4ef6-925a-5f75c28fa4f0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:18.854099 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-953fd14f-b8eb-4ef6-925a-5f75c28fa4f0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:18.860431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-953fd14f-b8eb-4ef6-925a-5f75c28fa4f0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:18.860950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-953fd14f-b8eb-4ef6-925a-5f75c28fa4f0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:18.861715 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-953fd14f-b8eb-4ef6-925a-5f75c28fa4f0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:18.862151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-953fd14f-b8eb-4ef6-925a-5f75c28fa4f0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:19.003765 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-953fd14f-b8eb-4ef6-925a-5f75c28fa4f0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:19.011784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-953fd14f-b8eb-4ef6-925a-5f75c28fa4f0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1757 microversion: 2.1 time: 0.162219 Jun 10 13:45:19.012353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1671/2585] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:18 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1757 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:45:20.028802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-58d66282-a8a5-41ce-91e3-7cc8837c2fc0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:20.029561 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-58d66282-a8a5-41ce-91e3-7cc8837c2fc0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:20.035554 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58d66282-a8a5-41ce-91e3-7cc8837c2fc0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:20.036545 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58d66282-a8a5-41ce-91e3-7cc8837c2fc0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:20.037339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58d66282-a8a5-41ce-91e3-7cc8837c2fc0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:20.037769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-58d66282-a8a5-41ce-91e3-7cc8837c2fc0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:20.182692 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-58d66282-a8a5-41ce-91e3-7cc8837c2fc0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:20.190574 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-58d66282-a8a5-41ce-91e3-7cc8837c2fc0 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1757 microversion: 2.1 time: 0.165746 Jun 10 13:45:20.191127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1672/2586] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:20 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1757 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:21.207590 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-5ee2bb90-4da4-42bb-8ac9-408b1d2acb86 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:21.208202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-5ee2bb90-4da4-42bb-8ac9-408b1d2acb86 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:21.214696 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5ee2bb90-4da4-42bb-8ac9-408b1d2acb86 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:21.215150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5ee2bb90-4da4-42bb-8ac9-408b1d2acb86 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:21.215910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5ee2bb90-4da4-42bb-8ac9-408b1d2acb86 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:21.216348 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-5ee2bb90-4da4-42bb-8ac9-408b1d2acb86 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:21.357941 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-5ee2bb90-4da4-42bb-8ac9-408b1d2acb86 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:21.366472 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-5ee2bb90-4da4-42bb-8ac9-408b1d2acb86 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1757 microversion: 2.1 time: 0.162724 Jun 10 13:45:21.367038 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1673/2587] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:21 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1757 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:22.383380 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e9874a4b-ad3f-47d1-8b4a-39a3554ac038 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:22.384001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e9874a4b-ad3f-47d1-8b4a-39a3554ac038 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:22.390428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9874a4b-ad3f-47d1-8b4a-39a3554ac038 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:22.390896 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9874a4b-ad3f-47d1-8b4a-39a3554ac038 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:22.391643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9874a4b-ad3f-47d1-8b4a-39a3554ac038 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:22.392070 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9874a4b-ad3f-47d1-8b4a-39a3554ac038 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:22.538190 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e9874a4b-ad3f-47d1-8b4a-39a3554ac038 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:22.546032 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e9874a4b-ad3f-47d1-8b4a-39a3554ac038 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1760 microversion: 2.1 time: 0.166489 Jun 10 13:45:22.546598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1674/2588] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:22 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1760 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:50.294713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1458555662", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-700124936", "security_groups": [{"name": "tempest-secgroup-smoke-83722939"}], "block_device_mapping_v2": [{"uuid": "127bac2c-5eb0-46e4-b16e-4cb34b0d2f11", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "6241abbb-4625-42fc-b64c-7f8fd8df7647"}]}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:45:50.303243 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=26 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:45:50.303918 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=82,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:45:50.410744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.network.neutron [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] validate_networks() for [('6241abbb-4625-42fc-b64c-7f8fd8df7647', None, None, None, None, None)] {{(pid=118092) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} Jun 10 13:45:50.818566 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Flavor limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jun 10 13:45:50.818995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Image limits 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jun 10 13:45:50.819452 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Flavor pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jun 10 13:45:50.819849 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Image pref 0:0:0 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jun 10 13:45:50.820253 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.virt.hardware [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118092) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jun 10 13:45:50.829217 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:50.829660 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:50.830440 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:50.830863 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:50.849083 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Getting quotas for project 372f8b4d0d5c4046ad91bea6d709a6ef. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jun 10 13:45:50.853048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.quota [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Getting quotas for user bf961149e78d4c0c8acb58f1cec447bd and project 372f8b4d0d5c4046ad91bea6d709a6ef. Resources: {'instances', 'ram', 'cores'} {{(pid=118092) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jun 10 13:45:50.908630 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Going to run 1 instances... {{(pid=118092) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jun 10 13:45:50.910781 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='127bac2c-5eb0-46e4-b16e-4cb34b0d2f11',volume_size=None,volume_type=None)] {{(pid=118092) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jun 10 13:45:51.319741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e495bf52-13d5-42c7-baef-c9afa3928cf9 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 1.028759 Jun 10 13:45:51.320904 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1675/2589] 10.222.0.25 () {60 vars in 1146 bytes} [Fri Jun 10 13:45:50 2022] POST /compute/v2.1/servers => generated 404 bytes in 1030 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jun 10 13:45:51.331047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d92065db-c03b-4cc3-9852-8a1824a4951e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:51.339744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d92065db-c03b-4cc3-9852-8a1824a4951e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:51.388611 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d92065db-c03b-4cc3-9852-8a1824a4951e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:51.395290 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d92065db-c03b-4cc3-9852-8a1824a4951e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1094 microversion: 2.1 time: 0.068735 Jun 10 13:45:51.395860 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1676/2590] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:51 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1094 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:52.411557 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-77581d7c-4f4f-4d25-a8b9-54075db4d42e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:52.412169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-77581d7c-4f4f-4d25-a8b9-54075db4d42e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:52.418695 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-77581d7c-4f4f-4d25-a8b9-54075db4d42e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:52.419147 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-77581d7c-4f4f-4d25-a8b9-54075db4d42e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:52.419910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-77581d7c-4f4f-4d25-a8b9-54075db4d42e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:52.420341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-77581d7c-4f4f-4d25-a8b9-54075db4d42e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:52.482283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-77581d7c-4f4f-4d25-a8b9-54075db4d42e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:52.490050 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-77581d7c-4f4f-4d25-a8b9-54075db4d42e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1144 microversion: 2.1 time: 0.082352 Jun 10 13:45:52.490606 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1677/2591] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:52 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1144 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:53.506467 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-160c67f9-37b1-496a-9b0c-387f84347389 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:53.507304 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-160c67f9-37b1-496a-9b0c-387f84347389 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:53.514158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-160c67f9-37b1-496a-9b0c-387f84347389 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:53.514682 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-160c67f9-37b1-496a-9b0c-387f84347389 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:53.515550 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-160c67f9-37b1-496a-9b0c-387f84347389 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:53.516059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-160c67f9-37b1-496a-9b0c-387f84347389 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:53.591975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-160c67f9-37b1-496a-9b0c-387f84347389 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:53.600096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-160c67f9-37b1-496a-9b0c-387f84347389 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1210 microversion: 2.1 time: 0.097747 Jun 10 13:45:53.600994 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1678/2592] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:53 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1210 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:54.616923 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ab9249a3-b097-46ea-8a83-a219b2229c09 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:54.617656 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ab9249a3-b097-46ea-8a83-a219b2229c09 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:54.624049 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ab9249a3-b097-46ea-8a83-a219b2229c09 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:54.624564 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ab9249a3-b097-46ea-8a83-a219b2229c09 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:54.625544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ab9249a3-b097-46ea-8a83-a219b2229c09 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:54.626023 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ab9249a3-b097-46ea-8a83-a219b2229c09 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:54.759114 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-ab9249a3-b097-46ea-8a83-a219b2229c09 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:54.766728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ab9249a3-b097-46ea-8a83-a219b2229c09 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1276 microversion: 2.1 time: 0.153867 Jun 10 13:45:54.767299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1679/2593] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:54 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1276 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:55.119417 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-f5cf74ba-66ef-4520-9f69-1b6b745eb7ca req-da83011b-12b4-45cd-b5a3-4650e7daf0dc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cae669f3-f80e-46d6-a535-49b4e5619a40", "tag": "5f0c1d8f-af9e-4aa5-8704-dd1c76f0d99c"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:45:55.133869 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-f5cf74ba-66ef-4520-9f69-1b6b745eb7ca req-da83011b-12b4-45cd-b5a3-4650e7daf0dc service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:55.134319 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-f5cf74ba-66ef-4520-9f69-1b6b745eb7ca req-da83011b-12b4-45cd-b5a3-4650e7daf0dc service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:55.148638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-f5cf74ba-66ef-4520-9f69-1b6b745eb7ca req-da83011b-12b4-45cd-b5a3-4650e7daf0dc service nova] Creating event network-changed:5f0c1d8f-af9e-4aa5-8704-dd1c76f0d99c for instance cae669f3-f80e-46d6-a535-49b4e5619a40 on hv-cinder-maste Jun 10 13:45:55.154123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-f5cf74ba-66ef-4520-9f69-1b6b745eb7ca req-da83011b-12b4-45cd-b5a3-4650e7daf0dc service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039364 Jun 10 13:45:55.154730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1680/2594] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:45:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:45:55.784370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0266b1aa-4491-4841-b988-c8323b258bde tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:55.785122 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0266b1aa-4491-4841-b988-c8323b258bde tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:55.791795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0266b1aa-4491-4841-b988-c8323b258bde tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:55.792244 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0266b1aa-4491-4841-b988-c8323b258bde tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:55.793030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0266b1aa-4491-4841-b988-c8323b258bde tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:55.793470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0266b1aa-4491-4841-b988-c8323b258bde tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:55.924483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-0266b1aa-4491-4841-b988-c8323b258bde tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:55.932602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0266b1aa-4491-4841-b988-c8323b258bde tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1264 microversion: 2.1 time: 0.152824 Jun 10 13:45:55.933250 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1681/2595] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:55 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1264 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:56.949855 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-384a4a2f-504e-4a45-aad0-9527513fb1eb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:56.950483 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-384a4a2f-504e-4a45-aad0-9527513fb1eb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:56.957484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-384a4a2f-504e-4a45-aad0-9527513fb1eb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:56.957953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-384a4a2f-504e-4a45-aad0-9527513fb1eb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:56.958704 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-384a4a2f-504e-4a45-aad0-9527513fb1eb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:56.959134 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-384a4a2f-504e-4a45-aad0-9527513fb1eb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:57.088123 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-384a4a2f-504e-4a45-aad0-9527513fb1eb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:57.095809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-384a4a2f-504e-4a45-aad0-9527513fb1eb tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1264 microversion: 2.1 time: 0.150096 Jun 10 13:45:57.102589 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1682/2596] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:56 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1264 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:45:57.763552 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c5fce932-f7dd-493f-923b-f9aba833b9e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-2085704531"}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:45:57.764080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c5fce932-f7dd-493f-923b-f9aba833b9e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:57.770006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5fce932-f7dd-493f-923b-f9aba833b9e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:57.770970 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5fce932-f7dd-493f-923b-f9aba833b9e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:57.771744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5fce932-f7dd-493f-923b-f9aba833b9e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:57.772169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c5fce932-f7dd-493f-923b-f9aba833b9e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:57.910598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.network.security_group_api [None req-c5fce932-f7dd-493f-923b-f9aba833b9e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Removing security group e39e3bb1-996e-4953-a1f1-efa4b62851ec from port 2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a Jun 10 13:45:58.532168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c5fce932-f7dd-493f-923b-f9aba833b9e6 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "POST /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/action" status: 202 len: 0 microversion: 2.1 time: 0.772429 Jun 10 13:45:58.533806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1683/2597] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Jun 10 13:45:57 2022] POST /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/action => generated 0 bytes in 775 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:45:58.541004 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1c9eb7b7-97c6-46d0-8024-6a100a90e4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:58.541110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1c9eb7b7-97c6-46d0-8024-6a100a90e4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:58.547402 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c9eb7b7-97c6-46d0-8024-6a100a90e4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:58.548086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c9eb7b7-97c6-46d0-8024-6a100a90e4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:58.549262 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c9eb7b7-97c6-46d0-8024-6a100a90e4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:58.549867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1c9eb7b7-97c6-46d0-8024-6a100a90e4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:58.722528 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-1c9eb7b7-97c6-46d0-8024-6a100a90e4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:58.731060 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1c9eb7b7-97c6-46d0-8024-6a100a90e4bd tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1264 microversion: 2.1 time: 0.195803 Jun 10 13:45:58.731700 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1684/2598] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:58 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1264 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:45:58.740372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-2fdb7718-a03d-4773-913b-b568a14a9d39 req-ab326345-899b-4827-bcc7-fca79851cdcf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "80cb1520-6b76-4212-82d5-b5ba482b481f", "tag": "2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:45:58.748946 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-2fdb7718-a03d-4773-913b-b568a14a9d39 req-ab326345-899b-4827-bcc7-fca79851cdcf service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:58.749809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-2fdb7718-a03d-4773-913b-b568a14a9d39 req-ab326345-899b-4827-bcc7-fca79851cdcf service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:58.768092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-2fdb7718-a03d-4773-913b-b568a14a9d39 req-ab326345-899b-4827-bcc7-fca79851cdcf service nova] Creating event network-changed:2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a for instance 80cb1520-6b76-4212-82d5-b5ba482b481f on hv-cinder-maste Jun 10 13:45:58.776394 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-2fdb7718-a03d-4773-913b-b568a14a9d39 req-ab326345-899b-4827-bcc7-fca79851cdcf service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041907 Jun 10 13:45:58.777438 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1685/2599] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:45:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:45:59.747988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-63cd14f1-883e-423f-aac7-eed5cb14b1de tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:45:59.748720 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-63cd14f1-883e-423f-aac7-eed5cb14b1de tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:45:59.755403 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63cd14f1-883e-423f-aac7-eed5cb14b1de tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:59.755851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63cd14f1-883e-423f-aac7-eed5cb14b1de tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:59.756617 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63cd14f1-883e-423f-aac7-eed5cb14b1de tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:45:59.757079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-63cd14f1-883e-423f-aac7-eed5cb14b1de tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:45:59.978682 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-63cd14f1-883e-423f-aac7-eed5cb14b1de tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:45:59.986166 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-63cd14f1-883e-423f-aac7-eed5cb14b1de tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1264 microversion: 2.1 time: 0.242442 Jun 10 13:45:59.986732 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1686/2600] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:45:59 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1264 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:46:00.112784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-8a12d142-07cc-4525-b88c-a1627663c548 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:00.113333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-8a12d142-07cc-4525-b88c-a1627663c548 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:00.124716 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a12d142-07cc-4525-b88c-a1627663c548 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:00.125212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a12d142-07cc-4525-b88c-a1627663c548 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:00.125995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a12d142-07cc-4525-b88c-a1627663c548 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:00.126430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-8a12d142-07cc-4525-b88c-a1627663c548 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:00.209820 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-8a12d142-07cc-4525-b88c-a1627663c548 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lazy-loading 'services' on Instance uuid 80cb1520-6b76-4212-82d5-b5ba482b481f {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:46:00.471516 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-8a12d142-07cc-4525-b88c-a1627663c548 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "DELETE /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments/d8298772-9338-4b96-b09c-db39371e8088" status: 202 len: 0 microversion: 2.1 time: 0.362613 Jun 10 13:46:00.472152 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1687/2601] 10.222.0.25 () {58 vars in 1415 bytes} [Fri Jun 10 13:46:00 2022] DELETE /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments/d8298772-9338-4b96-b09c-db39371e8088 => generated 0 bytes in 364 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:46:00.482202 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c3110bac-ec51-4f5f-b774-75d66204b3c3 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:00.486935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c3110bac-ec51-4f5f-b774-75d66204b3c3 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:00.490560 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3110bac-ec51-4f5f-b774-75d66204b3c3 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:00.491020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3110bac-ec51-4f5f-b774-75d66204b3c3 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:00.491803 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3110bac-ec51-4f5f-b774-75d66204b3c3 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:00.492247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c3110bac-ec51-4f5f-b774-75d66204b3c3 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:00.523339 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c3110bac-ec51-4f5f-b774-75d66204b3c3 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046275 Jun 10 13:46:00.523909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1688/2602] 10.222.0.25 () {58 vars in 1301 bytes} [Fri Jun 10 13:46:00 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:46:00.812508 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-e3851bff-653c-43f9-8970-6f6c0d82d155 req-996fb554-2f05-4b96-8175-1fe0b1273f6f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "80cb1520-6b76-4212-82d5-b5ba482b481f", "tag": "2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:46:00.821408 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-e3851bff-653c-43f9-8970-6f6c0d82d155 req-996fb554-2f05-4b96-8175-1fe0b1273f6f service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:00.821851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-e3851bff-653c-43f9-8970-6f6c0d82d155 req-996fb554-2f05-4b96-8175-1fe0b1273f6f service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:00.833809 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-e3851bff-653c-43f9-8970-6f6c0d82d155 req-996fb554-2f05-4b96-8175-1fe0b1273f6f service nova] Creating event network-changed:2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a for instance 80cb1520-6b76-4212-82d5-b5ba482b481f on hv-cinder-maste Jun 10 13:46:00.838649 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-e3851bff-653c-43f9-8970-6f6c0d82d155 req-996fb554-2f05-4b96-8175-1fe0b1273f6f service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029784 Jun 10 13:46:00.839460 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1689/2603] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:46:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:46:01.003651 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-914bf7f2-882a-46c2-b309-31546649d8f5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:01.004465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-914bf7f2-882a-46c2-b309-31546649d8f5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:01.007166 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:46:01.012601 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-914bf7f2-882a-46c2-b309-31546649d8f5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:01.013170 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-914bf7f2-882a-46c2-b309-31546649d8f5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:01.014066 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-914bf7f2-882a-46c2-b309-31546649d8f5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:01.014572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-914bf7f2-882a-46c2-b309-31546649d8f5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:01.202176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-914bf7f2-882a-46c2-b309-31546649d8f5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:46:01.210073 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-914bf7f2-882a-46c2-b309-31546649d8f5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1264 microversion: 2.1 time: 0.210931 Jun 10 13:46:01.210639 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1690/2604] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:00 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1264 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:46:01.535111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ac83aa77-0000-47e8-9cbe-74296b272272 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:01.536019 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-ac83aa77-0000-47e8-9cbe-74296b272272 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:01.537881 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell0:SELECT=8 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:46:01.542602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ac83aa77-0000-47e8-9cbe-74296b272272 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:01.543044 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ac83aa77-0000-47e8-9cbe-74296b272272 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:01.543841 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ac83aa77-0000-47e8-9cbe-74296b272272 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:01.544267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-ac83aa77-0000-47e8-9cbe-74296b272272 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:01.572913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ac83aa77-0000-47e8-9cbe-74296b272272 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041540 Jun 10 13:46:01.573462 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1691/2605] 10.222.0.25 () {58 vars in 1301 bytes} [Fri Jun 10 13:46:01 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:46:02.227664 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3572ab60-5baa-4089-b389-c41dfe75cf79 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:02.228338 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3572ab60-5baa-4089-b389-c41dfe75cf79 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:02.235194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3572ab60-5baa-4089-b389-c41dfe75cf79 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:02.235641 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3572ab60-5baa-4089-b389-c41dfe75cf79 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:02.236646 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3572ab60-5baa-4089-b389-c41dfe75cf79 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:02.237113 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3572ab60-5baa-4089-b389-c41dfe75cf79 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:02.428995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3572ab60-5baa-4089-b389-c41dfe75cf79 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:46:02.436504 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3572ab60-5baa-4089-b389-c41dfe75cf79 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1264 microversion: 2.1 time: 0.213346 Jun 10 13:46:02.437112 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1692/2606] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:02 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1264 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:46:02.585488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c6a3b1fa-3198-433a-9c35-38e7f9b25708 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:02.586411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c6a3b1fa-3198-433a-9c35-38e7f9b25708 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:02.592373 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6a3b1fa-3198-433a-9c35-38e7f9b25708 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:02.592851 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6a3b1fa-3198-433a-9c35-38e7f9b25708 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:02.593605 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6a3b1fa-3198-433a-9c35-38e7f9b25708 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:02.594043 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c6a3b1fa-3198-433a-9c35-38e7f9b25708 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:02.621854 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c6a3b1fa-3198-433a-9c35-38e7f9b25708 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041041 Jun 10 13:46:02.622410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1693/2607] 10.222.0.25 () {58 vars in 1301 bytes} [Fri Jun 10 13:46:02 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:46:03.453894 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3e1b42f3-afd3-4b93-9005-2cf593834ce5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:03.454513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3e1b42f3-afd3-4b93-9005-2cf593834ce5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:03.461532 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3e1b42f3-afd3-4b93-9005-2cf593834ce5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:03.461984 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3e1b42f3-afd3-4b93-9005-2cf593834ce5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:03.462741 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3e1b42f3-afd3-4b93-9005-2cf593834ce5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:03.463177 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3e1b42f3-afd3-4b93-9005-2cf593834ce5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:03.592362 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3e1b42f3-afd3-4b93-9005-2cf593834ce5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:46:03.595824 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:46:03.600722 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3e1b42f3-afd3-4b93-9005-2cf593834ce5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1264 microversion: 2.1 time: 0.150631 Jun 10 13:46:03.601330 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1694/2608] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:03 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1264 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:46:03.632533 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-68579d70-ca60-49db-97b0-03ec2eb7a384 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:03.633473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-68579d70-ca60-49db-97b0-03ec2eb7a384 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:03.640077 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68579d70-ca60-49db-97b0-03ec2eb7a384 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:03.640513 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68579d70-ca60-49db-97b0-03ec2eb7a384 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:03.641283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68579d70-ca60-49db-97b0-03ec2eb7a384 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:03.641713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-68579d70-ca60-49db-97b0-03ec2eb7a384 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:03.670379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-68579d70-ca60-49db-97b0-03ec2eb7a384 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041742 Jun 10 13:46:03.670914 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1695/2609] 10.222.0.25 () {58 vars in 1301 bytes} [Fri Jun 10 13:46:03 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jun 10 13:46:03.679925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1a617d24-5263-4fc7-a568-538afe858051 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:03.680397 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-1a617d24-5263-4fc7-a568-538afe858051 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:03.685972 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a617d24-5263-4fc7-a568-538afe858051 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:03.686411 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a617d24-5263-4fc7-a568-538afe858051 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:03.687150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a617d24-5263-4fc7-a568-538afe858051 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:03.687572 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-1a617d24-5263-4fc7-a568-538afe858051 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:03.759336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-1a617d24-5263-4fc7-a568-538afe858051 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] HTTP exception thrown: Instance 80cb1520-6b76-4212-82d5-b5ba482b481f is not attached to volume d8298772-9338-4b96-b09c-db39371e8088 Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1a617d24-5263-4fc7-a568-538afe858051 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Request method failure captured: Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments/d8298772-9338-4b96-b09c-db39371e8088 HTTP/1.1 Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0qU18vbY0Es3e7J0QJ6qXjTifdaESrw4XZ8_GYxdPKRLx26dET1SRVjSKfwvoazELeZKbzT2_IkiroIOhv98U1RyXecdV8TArBEJplF4xeATyHSbU8y0hV0jJCeerEhn1cCsWvnFbND413omcUAyBpkJIaEm1YAtYnm9UfbZ06PjO0 Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: ea8e34865f774cdf830b60908416ee9a Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-TestMinimumBasicScenario-597542981 Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:46:03.761669 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:46:03.762975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ea8e34865f774cdf830b60908416ee9a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ea8e34865f774cdf830b60908416ee9a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ea8e34865f774cdf830b60908416ee9a", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ea8e34865f774cdf830b60908416ee9a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:46:03.762975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-TestMinimumBasicScenario-597542981 Jun 10 13:46:03.762975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: ea8e34865f774cdf830b60908416ee9a Jun 10 13:46:03.762975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-TestMinimumBasicScenario-597542981 Jun 10 13:46:03.762975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-TestMinimumBasicScenario-597542981-project Jun 10 13:46:03.762975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:46:03.762975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:46:03.762975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: ad44d340bc1d4cf48694416948333102 Jun 10 13:46:03.762975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-TestMinimumBasicScenario-597542981-project Jun 10 13:46:03.762975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:46:03.762975 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 80cb1520-6b76-4212-82d5-b5ba482b481f is not attached to volume d8298772-9338-4b96-b09c-db39371e8088 Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'server_id': '80cb1520-6b76-4212-82d5-b5ba482b481f', 'id': 'd8298772-9338-4b96-b09c-db39371e8088'} Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 281, in get_by_volume_and_instance Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id d8298772-9338-4b96-b09c-db39371e8088. Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:46:03.763814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 80cb1520-6b76-4212-82d5-b5ba482b481f is not attached to volume d8298772-9338-4b96-b09c-db39371e8088 Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 80cb1520-6b76-4212-82d5-b5ba482b481f is not attached to volume d8298772-9338-4b96-b09c-db39371e8088 Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:46:03.765002 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1a617d24-5263-4fc7-a568-538afe858051 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Returning 404 to user: Instance 80cb1520-6b76-4212-82d5-b5ba482b481f is not attached to volume d8298772-9338-4b96-b09c-db39371e8088 {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:46:03.765950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1a617d24-5263-4fc7-a568-538afe858051 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "DELETE /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments/d8298772-9338-4b96-b09c-db39371e8088" status: 404 len: 154 microversion: 2.1 time: 0.086745 Jun 10 13:46:03.765950 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1696/2610] 10.222.0.25 () {58 vars in 1415 bytes} [Fri Jun 10 13:46:03 2022] DELETE /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f/os-volume_attachments/d8298772-9338-4b96-b09c-db39371e8088 => generated 154 bytes in 88 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:46:04.617921 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9e3abde2-2f23-4c53-9ac2-417afda5552c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:04.618541 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-9e3abde2-2f23-4c53-9ac2-417afda5552c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:04.625138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e3abde2-2f23-4c53-9ac2-417afda5552c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:04.625585 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e3abde2-2f23-4c53-9ac2-417afda5552c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:04.626381 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e3abde2-2f23-4c53-9ac2-417afda5552c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:04.626801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-9e3abde2-2f23-4c53-9ac2-417afda5552c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:04.757280 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-9e3abde2-2f23-4c53-9ac2-417afda5552c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:46:04.764674 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9e3abde2-2f23-4c53-9ac2-417afda5552c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1264 microversion: 2.1 time: 0.151235 Jun 10 13:46:04.765270 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1697/2611] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:04 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1264 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:46:05.015492 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b943dcb1-f6de-4122-8dbf-d5c3761b1e68 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:05.015991 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b943dcb1-f6de-4122-8dbf-d5c3761b1e68 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:05.022444 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b943dcb1-f6de-4122-8dbf-d5c3761b1e68 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:05.022893 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b943dcb1-f6de-4122-8dbf-d5c3761b1e68 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:05.023650 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b943dcb1-f6de-4122-8dbf-d5c3761b1e68 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:05.024087 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b943dcb1-f6de-4122-8dbf-d5c3761b1e68 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:05.049169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b943dcb1-f6de-4122-8dbf-d5c3761b1e68 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:46:05.352100 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b943dcb1-f6de-4122-8dbf-d5c3761b1e68 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "DELETE /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 204 len: 0 microversion: 2.1 time: 0.340194 Jun 10 13:46:05.352684 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1698/2612] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:46:05 2022] DELETE /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 0 bytes in 342 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jun 10 13:46:05.360978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-82721fd9-a49e-4314-bc26-c879578d0320 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:05.361662 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-82721fd9-a49e-4314-bc26-c879578d0320 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:05.369377 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-82721fd9-a49e-4314-bc26-c879578d0320 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:05.369846 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-82721fd9-a49e-4314-bc26-c879578d0320 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:05.370679 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-82721fd9-a49e-4314-bc26-c879578d0320 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:05.371126 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-82721fd9-a49e-4314-bc26-c879578d0320 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:05.501336 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-82721fd9-a49e-4314-bc26-c879578d0320 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:46:05.508670 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-82721fd9-a49e-4314-bc26-c879578d0320 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1555 microversion: 2.1 time: 0.151650 Jun 10 13:46:05.509259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1699/2613] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:05 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1555 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:46:05.781215 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-af781039-88aa-4817-a672-06af9bbd7ac5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:05.781876 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-af781039-88aa-4817-a672-06af9bbd7ac5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:05.788242 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-af781039-88aa-4817-a672-06af9bbd7ac5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:05.788691 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-af781039-88aa-4817-a672-06af9bbd7ac5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:05.789474 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-af781039-88aa-4817-a672-06af9bbd7ac5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:05.789898 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-af781039-88aa-4817-a672-06af9bbd7ac5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:05.916209 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-af781039-88aa-4817-a672-06af9bbd7ac5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:46:05.923721 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-af781039-88aa-4817-a672-06af9bbd7ac5 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1264 microversion: 2.1 time: 0.146255 Jun 10 13:46:05.924289 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1700/2614] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:05 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1264 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:46:06.525630 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-99c581cf-79a0-4895-ae64-ff26547e3b6f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:06.526247 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-99c581cf-79a0-4895-ae64-ff26547e3b6f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:06.532385 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99c581cf-79a0-4895-ae64-ff26547e3b6f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:06.532872 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99c581cf-79a0-4895-ae64-ff26547e3b6f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:06.533638 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99c581cf-79a0-4895-ae64-ff26547e3b6f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:06.534080 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-99c581cf-79a0-4895-ae64-ff26547e3b6f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:06.664988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-99c581cf-79a0-4895-ae64-ff26547e3b6f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:46:06.672451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-99c581cf-79a0-4895-ae64-ff26547e3b6f tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1555 microversion: 2.1 time: 0.150733 Jun 10 13:46:06.673056 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1701/2615] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:06 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1555 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:46:06.732659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-436b5dc5-c731-448b-8b4e-28c82d4a8909 req-bd93e6c4-3dd3-4790-a6eb-9532aa95d169 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cae669f3-f80e-46d6-a535-49b4e5619a40", "name": "network-vif-plugged", "status": "completed", "tag": "5f0c1d8f-af9e-4aa5-8704-dd1c76f0d99c"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:46:06.741315 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-436b5dc5-c731-448b-8b4e-28c82d4a8909 req-bd93e6c4-3dd3-4790-a6eb-9532aa95d169 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:06.741759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-436b5dc5-c731-448b-8b4e-28c82d4a8909 req-bd93e6c4-3dd3-4790-a6eb-9532aa95d169 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:06.753035 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-436b5dc5-c731-448b-8b4e-28c82d4a8909 req-bd93e6c4-3dd3-4790-a6eb-9532aa95d169 service nova] Creating event network-vif-plugged:5f0c1d8f-af9e-4aa5-8704-dd1c76f0d99c for instance cae669f3-f80e-46d6-a535-49b4e5619a40 on hv-cinder-maste Jun 10 13:46:06.758482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-436b5dc5-c731-448b-8b4e-28c82d4a8909 req-bd93e6c4-3dd3-4790-a6eb-9532aa95d169 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.084261 Jun 10 13:46:06.759071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1702/2616] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:46:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:46:06.941977 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-98149e1d-3d72-4a8b-a028-67fb8f2ea15f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:06.942706 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-98149e1d-3d72-4a8b-a028-67fb8f2ea15f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:06.949191 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98149e1d-3d72-4a8b-a028-67fb8f2ea15f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:06.949688 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98149e1d-3d72-4a8b-a028-67fb8f2ea15f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:06.950449 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98149e1d-3d72-4a8b-a028-67fb8f2ea15f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:06.950888 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-98149e1d-3d72-4a8b-a028-67fb8f2ea15f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:07.083596 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-98149e1d-3d72-4a8b-a028-67fb8f2ea15f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:46:07.091635 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-98149e1d-3d72-4a8b-a028-67fb8f2ea15f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1264 microversion: 2.1 time: 0.154188 Jun 10 13:46:07.092205 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1703/2617] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:06 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1264 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:46:07.690105 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dbee2180-6fa9-4ea2-aa84-22d6a0e8ff3d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:07.690847 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-dbee2180-6fa9-4ea2-aa84-22d6a0e8ff3d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:07.696868 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dbee2180-6fa9-4ea2-aa84-22d6a0e8ff3d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:07.697321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dbee2180-6fa9-4ea2-aa84-22d6a0e8ff3d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:07.698350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dbee2180-6fa9-4ea2-aa84-22d6a0e8ff3d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:07.698784 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-dbee2180-6fa9-4ea2-aa84-22d6a0e8ff3d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:07.887347 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-dbee2180-6fa9-4ea2-aa84-22d6a0e8ff3d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:46:07.894773 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dbee2180-6fa9-4ea2-aa84-22d6a0e8ff3d tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1555 microversion: 2.1 time: 0.208600 Jun 10 13:46:07.897744 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1704/2618] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:07 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1555 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:46:08.075633 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-62620567-4517-4290-92ee-cd539f85820a req-6277d2dc-30bf-46cc-a115-8efc4f88c6bc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "80cb1520-6b76-4212-82d5-b5ba482b481f", "name": "network-vif-deleted", "tag": "2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:46:08.084037 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-62620567-4517-4290-92ee-cd539f85820a req-6277d2dc-30bf-46cc-a115-8efc4f88c6bc service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:08.084477 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-62620567-4517-4290-92ee-cd539f85820a req-6277d2dc-30bf-46cc-a115-8efc4f88c6bc service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:08.095506 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-62620567-4517-4290-92ee-cd539f85820a req-6277d2dc-30bf-46cc-a115-8efc4f88c6bc service nova] Creating event network-vif-deleted:2692a5fd-ad9d-4c93-9fd0-c8f5c2db411a for instance 80cb1520-6b76-4212-82d5-b5ba482b481f on hv-cinder-maste Jun 10 13:46:08.100427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-62620567-4517-4290-92ee-cd539f85820a req-6277d2dc-30bf-46cc-a115-8efc4f88c6bc service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028918 Jun 10 13:46:08.101032 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1705/2619] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:46:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:46:08.108637 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c21f2b6e-f11c-46a9-ba14-0346fa759d10 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:08.109351 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-c21f2b6e-f11c-46a9-ba14-0346fa759d10 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:08.115935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c21f2b6e-f11c-46a9-ba14-0346fa759d10 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:08.116379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c21f2b6e-f11c-46a9-ba14-0346fa759d10 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:08.117156 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c21f2b6e-f11c-46a9-ba14-0346fa759d10 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:08.117584 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-c21f2b6e-f11c-46a9-ba14-0346fa759d10 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:08.244006 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-c21f2b6e-f11c-46a9-ba14-0346fa759d10 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:46:08.251350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c21f2b6e-f11c-46a9-ba14-0346fa759d10 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1264 microversion: 2.1 time: 0.146523 Jun 10 13:46:08.251924 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1706/2620] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:08 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1264 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:46:08.913053 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-47123551-da4c-4cd0-a017-58d0211893ab tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:08.913576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-47123551-da4c-4cd0-a017-58d0211893ab tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:08.919609 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-47123551-da4c-4cd0-a017-58d0211893ab tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:08.920047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-47123551-da4c-4cd0-a017-58d0211893ab tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:08.920814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-47123551-da4c-4cd0-a017-58d0211893ab tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:08.921245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-47123551-da4c-4cd0-a017-58d0211893ab tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:08.951138 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-47123551-da4c-4cd0-a017-58d0211893ab tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:08.951578 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-47123551-da4c-4cd0-a017-58d0211893ab tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:08.952014 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-47123551-da4c-4cd0-a017-58d0211893ab tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lazy-loading 'fault' on Instance uuid 80cb1520-6b76-4212-82d5-b5ba482b481f {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:46:08.995203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-47123551-da4c-4cd0-a017-58d0211893ab tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad44d340bc1d4cf48694416948333102', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea8e34865f774cdf830b60908416ee9a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:46:09.002920 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-47123551-da4c-4cd0-a017-58d0211893ab tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 200 len: 1349 microversion: 2.1 time: 0.094944 Jun 10 13:46:09.003484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1707/2621] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:08 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 1349 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:46:09.268544 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-be36b62c-2b9d-4134-814c-cb17ce7f67ee tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:09.269239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-be36b62c-2b9d-4134-814c-cb17ce7f67ee tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:09.275655 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be36b62c-2b9d-4134-814c-cb17ce7f67ee tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:09.276176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be36b62c-2b9d-4134-814c-cb17ce7f67ee tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:09.276953 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be36b62c-2b9d-4134-814c-cb17ce7f67ee tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:09.277536 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-be36b62c-2b9d-4134-814c-cb17ce7f67ee tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:09.407407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-be36b62c-2b9d-4134-814c-cb17ce7f67ee tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:46:09.415607 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-be36b62c-2b9d-4134-814c-cb17ce7f67ee tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1264 microversion: 2.1 time: 0.150930 Jun 10 13:46:09.416167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1708/2622] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:09 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1264 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:46:10.021233 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-96a78f0f-d006-4150-80c9-089f7152f0a8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:10.021840 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-96a78f0f-d006-4150-80c9-089f7152f0a8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] [instance: 80cb1520-6b76-4212-82d5-b5ba482b481f] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:10.027777 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96a78f0f-d006-4150-80c9-089f7152f0a8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:10.028221 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96a78f0f-d006-4150-80c9-089f7152f0a8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:10.028997 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96a78f0f-d006-4150-80c9-089f7152f0a8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:10.029428 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-96a78f0f-d006-4150-80c9-089f7152f0a8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:10.042201 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-96a78f0f-d006-4150-80c9-089f7152f0a8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] HTTP exception thrown: Instance 80cb1520-6b76-4212-82d5-b5ba482b481f could not be found. Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-96a78f0f-d006-4150-80c9-089f7152f0a8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Request method failure captured: Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f HTTP/1.1 Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0qU18vbY0Es3e7J0QJ6qXjTifdaESrw4XZ8_GYxdPKRLx26dET1SRVjSKfwvoazELeZKbzT2_IkiroIOhv98U1RyXecdV8TArBEJplF4xeATyHSbU8y0hV0jJCeerEhn1cCsWvnFbND413omcUAyBpkJIaEm1YAtYnm9UfbZ06PjO0 Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: ea8e34865f774cdf830b60908416ee9a Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-TestMinimumBasicScenario-597542981 Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:46:10.043801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:46:10.045148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ea8e34865f774cdf830b60908416ee9a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ea8e34865f774cdf830b60908416ee9a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ea8e34865f774cdf830b60908416ee9a", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ea8e34865f774cdf830b60908416ee9a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:46:10.045148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-TestMinimumBasicScenario-597542981 Jun 10 13:46:10.045148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: ea8e34865f774cdf830b60908416ee9a Jun 10 13:46:10.045148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-TestMinimumBasicScenario-597542981 Jun 10 13:46:10.045148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-TestMinimumBasicScenario-597542981-project Jun 10 13:46:10.045148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:46:10.045148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:46:10.045148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: ad44d340bc1d4cf48694416948333102 Jun 10 13:46:10.045148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-TestMinimumBasicScenario-597542981-project Jun 10 13:46:10.045148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:46:10.045148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 80cb1520-6b76-4212-82d5-b5ba482b481f could not be found. Jun 10 13:46:10.045148 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '80cb1520-6b76-4212-82d5-b5ba482b481f'} Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 80cb1520-6b76-4212-82d5-b5ba482b481f could not be found. Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:46:10.045955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 80cb1520-6b76-4212-82d5-b5ba482b481f could not be found. Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 80cb1520-6b76-4212-82d5-b5ba482b481f could not be found. Jun 10 13:46:10.047269 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:46:10.048341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-96a78f0f-d006-4150-80c9-089f7152f0a8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Returning 404 to user: Instance 80cb1520-6b76-4212-82d5-b5ba482b481f could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:46:10.048341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-96a78f0f-d006-4150-80c9-089f7152f0a8 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f" status: 404 len: 111 microversion: 2.1 time: 0.028492 Jun 10 13:46:10.048341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1709/2623] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:10 2022] GET /compute/v2.1/servers/80cb1520-6b76-4212-82d5-b5ba482b481f => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:46:10.053121 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c308ae08-7b85-4cb1-9bd0-05c81209b118 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] Calling method '.version_select at 0x7f93053393a0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:10.064111 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=3,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:46:10.160158 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c308ae08-7b85-4cb1-9bd0-05c81209b118 tempest-TestMinimumBasicScenario-597542981 tempest-TestMinimumBasicScenario-597542981-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1741328647" status: 202 len: 0 microversion: 2.1 time: 0.110349 Jun 10 13:46:10.160901 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1710/2624] 10.222.0.25 () {58 vars in 1271 bytes} [Fri Jun 10 13:46:10 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1741328647 => generated 0 bytes in 112 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:46:10.433300 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e7a5121d-cf70-44ea-903f-2bc01589ab2f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:10.433895 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e7a5121d-cf70-44ea-903f-2bc01589ab2f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:10.441203 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7a5121d-cf70-44ea-903f-2bc01589ab2f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:10.441652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7a5121d-cf70-44ea-903f-2bc01589ab2f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:10.442405 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7a5121d-cf70-44ea-903f-2bc01589ab2f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:10.443473 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e7a5121d-cf70-44ea-903f-2bc01589ab2f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:10.572321 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e7a5121d-cf70-44ea-903f-2bc01589ab2f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:46:10.580196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e7a5121d-cf70-44ea-903f-2bc01589ab2f tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1451 microversion: 2.1 time: 0.151650 Jun 10 13:46:10.580782 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1711/2625] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:10 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1451 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:46:10.595770 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-38cda311-1e3a-4850-b49d-b43027574b6c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:46:10.596371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-38cda311-1e3a-4850-b49d-b43027574b6c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:46:10.603333 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-38cda311-1e3a-4850-b49d-b43027574b6c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:10.603775 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-38cda311-1e3a-4850-b49d-b43027574b6c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:10.604535 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-38cda311-1e3a-4850-b49d-b43027574b6c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:10.604993 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-38cda311-1e3a-4850-b49d-b43027574b6c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:10.624861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:46:10.742188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-38cda311-1e3a-4850-b49d-b43027574b6c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:46:10.749371 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-38cda311-1e3a-4850-b49d-b43027574b6c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1451 microversion: 2.1 time: 0.157279 Jun 10 13:46:10.749933 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1712/2626] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:46:10 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1451 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:46:12.724668 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-58f23f2f-1ac9-49a9-a8ce-270eabd2b66a req-ba51e3b2-604b-4842-9840-77b171150fa9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cae669f3-f80e-46d6-a535-49b4e5619a40", "tag": "5f0c1d8f-af9e-4aa5-8704-dd1c76f0d99c"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:46:12.734335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-58f23f2f-1ac9-49a9-a8ce-270eabd2b66a req-ba51e3b2-604b-4842-9840-77b171150fa9 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:46:12.734917 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-58f23f2f-1ac9-49a9-a8ce-270eabd2b66a req-ba51e3b2-604b-4842-9840-77b171150fa9 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:46:12.749196 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-58f23f2f-1ac9-49a9-a8ce-270eabd2b66a req-ba51e3b2-604b-4842-9840-77b171150fa9 service nova] Creating event network-changed:5f0c1d8f-af9e-4aa5-8704-dd1c76f0d99c for instance cae669f3-f80e-46d6-a535-49b4e5619a40 on hv-cinder-maste Jun 10 13:46:12.754246 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-58f23f2f-1ac9-49a9-a8ce-270eabd2b66a req-ba51e3b2-604b-4842-9840-77b171150fa9 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.111301 Jun 10 13:46:12.755680 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1713/2627] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:46:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:47:52.396100 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR oslo.messaging._drivers.impl_rabbit [-] [28f4d62e-58f2-4fc0-963e-d17379b93d6e] AMQP server on 10.222.0.25:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed Jun 10 13:47:52.396965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=6 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:47:52.397712 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=4 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:47:52.402526 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a75d4d6f-68d3-4908-b23c-22e3227c67cf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:47:52.403110 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a75d4d6f-68d3-4908-b23c-22e3227c67cf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:47:52.409261 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a75d4d6f-68d3-4908-b23c-22e3227c67cf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:52.409715 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a75d4d6f-68d3-4908-b23c-22e3227c67cf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:52.410481 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a75d4d6f-68d3-4908-b23c-22e3227c67cf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:52.410965 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-a75d4d6f-68d3-4908-b23c-22e3227c67cf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:52.436024 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-a75d4d6f-68d3-4908-b23c-22e3227c67cf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:47:52.748451 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a75d4d6f-68d3-4908-b23c-22e3227c67cf tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "DELETE /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 204 len: 0 microversion: 2.1 time: 0.354046 Jun 10 13:47:52.749071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1714/2628] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:47:52 2022] DELETE /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jun 10 13:47:52.754308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-89c72d33-9e30-4f55-93c6-ab4613e2acdd req-700b9448-6b88-420c-9248-e6f494c8a9ea service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cae669f3-f80e-46d6-a535-49b4e5619a40", "tag": "5f0c1d8f-af9e-4aa5-8704-dd1c76f0d99c"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:47:52.762266 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-89c72d33-9e30-4f55-93c6-ab4613e2acdd req-700b9448-6b88-420c-9248-e6f494c8a9ea service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:52.762710 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-89c72d33-9e30-4f55-93c6-ab4613e2acdd req-700b9448-6b88-420c-9248-e6f494c8a9ea service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:52.773699 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-89c72d33-9e30-4f55-93c6-ab4613e2acdd req-700b9448-6b88-420c-9248-e6f494c8a9ea service nova] Creating event network-changed:5f0c1d8f-af9e-4aa5-8704-dd1c76f0d99c for instance cae669f3-f80e-46d6-a535-49b4e5619a40 on hv-cinder-maste Jun 10 13:47:52.778425 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-89c72d33-9e30-4f55-93c6-ab4613e2acdd req-700b9448-6b88-420c-9248-e6f494c8a9ea service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.027983 Jun 10 13:47:52.779007 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1715/2629] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:47:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:47:52.785879 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2aebbfa2-c13b-4a51-8248-6b26e3a9efa8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:47:52.786721 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2aebbfa2-c13b-4a51-8248-6b26e3a9efa8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:47:52.793326 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2aebbfa2-c13b-4a51-8248-6b26e3a9efa8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:52.793831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2aebbfa2-c13b-4a51-8248-6b26e3a9efa8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:52.794652 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2aebbfa2-c13b-4a51-8248-6b26e3a9efa8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:52.795079 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2aebbfa2-c13b-4a51-8248-6b26e3a9efa8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:53.036988 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2aebbfa2-c13b-4a51-8248-6b26e3a9efa8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:47:53.045366 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2aebbfa2-c13b-4a51-8248-6b26e3a9efa8 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1575 microversion: 2.1 time: 0.263374 Jun 10 13:47:53.046213 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1716/2630] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:47:52 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1575 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:47:54.063583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d3a6c652-3cd0-4984-a95e-8a4c7a410f4d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:47:54.064258 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-d3a6c652-3cd0-4984-a95e-8a4c7a410f4d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:47:54.075575 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d3a6c652-3cd0-4984-a95e-8a4c7a410f4d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:54.076022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d3a6c652-3cd0-4984-a95e-8a4c7a410f4d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:54.076814 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d3a6c652-3cd0-4984-a95e-8a4c7a410f4d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:54.077252 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-d3a6c652-3cd0-4984-a95e-8a4c7a410f4d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:54.097096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] [28f4d62e-58f2-4fc0-963e-d17379b93d6e] Reconnected to AMQP server on 10.222.0.25:5672 via [amqp] client with port 47466. Jun 10 13:47:54.262792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-d3a6c652-3cd0-4984-a95e-8a4c7a410f4d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:47:54.271553 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d3a6c652-3cd0-4984-a95e-8a4c7a410f4d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1575 microversion: 2.1 time: 0.211290 Jun 10 13:47:54.272370 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1717/2631] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:47:54 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1575 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:47:55.286021 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-b285fa3c-73b3-48f7-b11e-f70a06a3714e req-8d9fb23e-6719-4b3f-bc8f-a6bcd58e3b66 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cae669f3-f80e-46d6-a535-49b4e5619a40", "name": "network-vif-deleted", "tag": "5f0c1d8f-af9e-4aa5-8704-dd1c76f0d99c"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:47:55.294308 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-b285fa3c-73b3-48f7-b11e-f70a06a3714e req-8d9fb23e-6719-4b3f-bc8f-a6bcd58e3b66 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:55.294755 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-b285fa3c-73b3-48f7-b11e-f70a06a3714e req-8d9fb23e-6719-4b3f-bc8f-a6bcd58e3b66 service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:55.305690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-b285fa3c-73b3-48f7-b11e-f70a06a3714e req-8d9fb23e-6719-4b3f-bc8f-a6bcd58e3b66 service nova] Creating event network-vif-deleted:5f0c1d8f-af9e-4aa5-8704-dd1c76f0d99c for instance cae669f3-f80e-46d6-a535-49b4e5619a40 on hv-cinder-maste Jun 10 13:47:55.311124 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-b285fa3c-73b3-48f7-b11e-f70a06a3714e req-8d9fb23e-6719-4b3f-bc8f-a6bcd58e3b66 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028717 Jun 10 13:47:55.311760 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1718/2632] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:47:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:47:55.318613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e9f86197-ea81-4292-a50c-661efd3fd65d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:47:55.318613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e9f86197-ea81-4292-a50c-661efd3fd65d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:47:55.327045 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9f86197-ea81-4292-a50c-661efd3fd65d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:55.327515 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9f86197-ea81-4292-a50c-661efd3fd65d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:55.328273 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9f86197-ea81-4292-a50c-661efd3fd65d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:55.328801 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e9f86197-ea81-4292-a50c-661efd3fd65d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:55.393139 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-e9f86197-ea81-4292-a50c-661efd3fd65d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:47:55.400693 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e9f86197-ea81-4292-a50c-661efd3fd65d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1391 microversion: 2.1 time: 0.087839 Jun 10 13:47:55.401285 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1719/2633] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:47:55 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1391 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:47:56.416482 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b1a4f08a-5a02-4c1f-a9af-d93ef87fc44a tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:47:56.417127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b1a4f08a-5a02-4c1f-a9af-d93ef87fc44a tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:47:56.423976 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b1a4f08a-5a02-4c1f-a9af-d93ef87fc44a tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:56.424418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b1a4f08a-5a02-4c1f-a9af-d93ef87fc44a tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:56.425206 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b1a4f08a-5a02-4c1f-a9af-d93ef87fc44a tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:56.425632 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b1a4f08a-5a02-4c1f-a9af-d93ef87fc44a tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:56.458503 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b1a4f08a-5a02-4c1f-a9af-d93ef87fc44a tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:56.458867 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b1a4f08a-5a02-4c1f-a9af-d93ef87fc44a tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:56.459305 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-b1a4f08a-5a02-4c1f-a9af-d93ef87fc44a tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lazy-loading 'fault' on Instance uuid cae669f3-f80e-46d6-a535-49b4e5619a40 {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:47:56.501865 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-b1a4f08a-5a02-4c1f-a9af-d93ef87fc44a tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:47:56.509418 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b1a4f08a-5a02-4c1f-a9af-d93ef87fc44a tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 200 len: 1230 microversion: 2.1 time: 0.096718 Jun 10 13:47:56.510001 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1720/2634] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:47:56 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 1230 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:47:57.526016 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b421e0af-4313-4067-a7f2-5c9fdd8fb64c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:47:57.526608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-b421e0af-4313-4067-a7f2-5c9fdd8fb64c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: cae669f3-f80e-46d6-a535-49b4e5619a40] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:47:57.533106 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b421e0af-4313-4067-a7f2-5c9fdd8fb64c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:57.533547 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b421e0af-4313-4067-a7f2-5c9fdd8fb64c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:57.534291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b421e0af-4313-4067-a7f2-5c9fdd8fb64c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:47:57.534714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-b421e0af-4313-4067-a7f2-5c9fdd8fb64c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:47:57.547355 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-b421e0af-4313-4067-a7f2-5c9fdd8fb64c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] HTTP exception thrown: Instance cae669f3-f80e-46d6-a535-49b4e5619a40 could not be found. Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b421e0af-4313-4067-a7f2-5c9fdd8fb64c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Request method failure captured: Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 HTTP/1.1 Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0naMDwEZ-YRSB7u9MWmuxa7VCZvbw5ekhxDtysZCwuiejBDwGfJK6HGQuuDfJjuNsnxxtAcrbNyuDjyZEn2ArGkghDHQNGqM2deKRpOfTSFnRaPrcSQSdBg7vhdrLqh7OJInyc42VEgwet4IzwTw2sa1gRaoreRqelNgRh5zNivNGE Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 372f8b4d0d5c4046ad91bea6d709a6ef Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:47:57.548981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:47:57.557470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_372f8b4d0d5c4046ad91bea6d709a6ef", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:47:57.557470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:47:57.557470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 372f8b4d0d5c4046ad91bea6d709a6ef Jun 10 13:47:57.557470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:47:57.557470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-TestVolumeBootPattern-1295034315-project Jun 10 13:47:57.557470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:47:57.557470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:47:57.557470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: bf961149e78d4c0c8acb58f1cec447bd Jun 10 13:47:57.557470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-TestVolumeBootPattern-1295034315-project Jun 10 13:47:57.557470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:47:57.557470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance cae669f3-f80e-46d6-a535-49b4e5619a40 could not be found. Jun 10 13:47:57.557470 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'cae669f3-f80e-46d6-a535-49b4e5619a40'} Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cae669f3-f80e-46d6-a535-49b4e5619a40 could not be found. Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:47:57.559383 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cae669f3-f80e-46d6-a535-49b4e5619a40 could not be found. Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cae669f3-f80e-46d6-a535-49b4e5619a40 could not be found. Jun 10 13:47:57.560714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:47:57.562159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-b421e0af-4313-4067-a7f2-5c9fdd8fb64c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Returning 404 to user: Instance cae669f3-f80e-46d6-a535-49b4e5619a40 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:47:57.562159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-b421e0af-4313-4067-a7f2-5c9fdd8fb64c tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40" status: 404 len: 111 microversion: 2.1 time: 0.028133 Jun 10 13:47:57.562159 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1721/2635] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:47:57 2022] GET /compute/v2.1/servers/cae669f3-f80e-46d6-a535-49b4e5619a40 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:48:01.093047 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0ccddf7d-f038-41ed-a1df-88e4f776ad2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:01.093687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0ccddf7d-f038-41ed-a1df-88e4f776ad2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:48:01.099903 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ccddf7d-f038-41ed-a1df-88e4f776ad2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:01.100350 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ccddf7d-f038-41ed-a1df-88e4f776ad2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:01.101131 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ccddf7d-f038-41ed-a1df-88e4f776ad2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:01.101567 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0ccddf7d-f038-41ed-a1df-88e4f776ad2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:01.127185 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0ccddf7d-f038-41ed-a1df-88e4f776ad2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Going to try to terminate instance {{(pid=118092) delete /opt/stack/nova/nova/compute/api.py:2664}} Jun 10 13:48:01.434537 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0ccddf7d-f038-41ed-a1df-88e4f776ad2d tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "DELETE /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 204 len: 0 microversion: 2.1 time: 0.345031 Jun 10 13:48:01.435127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1722/2636] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:48:01 2022] DELETE /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 0 bytes in 347 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jun 10 13:48:01.444055 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-f8fdcaeb-a392-4107-8c97-69d5fefc2671 req-589fe22a-51da-4321-848a-d16e84a8445c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "53f34c67-563c-4a95-ac23-25e7d05d305d", "tag": "c60dc47c-8f9b-48bf-b642-4c2f82a01fe6"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:48:01.449620 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-f8fdcaeb-a392-4107-8c97-69d5fefc2671 req-589fe22a-51da-4321-848a-d16e84a8445c service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:01.452293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-f8fdcaeb-a392-4107-8c97-69d5fefc2671 req-589fe22a-51da-4321-848a-d16e84a8445c service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:01.463598 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-f8fdcaeb-a392-4107-8c97-69d5fefc2671 req-589fe22a-51da-4321-848a-d16e84a8445c service nova] Creating event network-changed:c60dc47c-8f9b-48bf-b642-4c2f82a01fe6 for instance 53f34c67-563c-4a95-ac23-25e7d05d305d on hv-cinder-maste Jun 10 13:48:01.468676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-f8fdcaeb-a392-4107-8c97-69d5fefc2671 req-589fe22a-51da-4321-848a-d16e84a8445c service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031522 Jun 10 13:48:01.469618 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1723/2637] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:48:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jun 10 13:48:01.480886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-f553deba-0b00-418a-a7ef-c3f9d844a787 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:01.480886 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-f553deba-0b00-418a-a7ef-c3f9d844a787 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:48:01.483020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f553deba-0b00-418a-a7ef-c3f9d844a787 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:01.483312 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f553deba-0b00-418a-a7ef-c3f9d844a787 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:01.484097 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f553deba-0b00-418a-a7ef-c3f9d844a787 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:01.484600 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-f553deba-0b00-418a-a7ef-c3f9d844a787 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:01.622376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-f553deba-0b00-418a-a7ef-c3f9d844a787 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:48:01.630376 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-f553deba-0b00-418a-a7ef-c3f9d844a787 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1575 microversion: 2.1 time: 0.159626 Jun 10 13:48:01.630955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1724/2638] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:48:01 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1575 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:48:02.647910 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-74714976-05cc-41f3-869d-41eadcc0812b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:02.648520 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-74714976-05cc-41f3-869d-41eadcc0812b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:48:02.656048 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74714976-05cc-41f3-869d-41eadcc0812b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:02.656493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74714976-05cc-41f3-869d-41eadcc0812b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:02.657297 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74714976-05cc-41f3-869d-41eadcc0812b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:02.657728 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-74714976-05cc-41f3-869d-41eadcc0812b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:02.786149 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-74714976-05cc-41f3-869d-41eadcc0812b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:48:02.793711 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-74714976-05cc-41f3-869d-41eadcc0812b tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1575 microversion: 2.1 time: 0.149730 Jun 10 13:48:02.794286 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1725/2639] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:48:02 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1575 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:48:03.792022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [req-d188caa3-1014-4f5a-821b-46749ea223af req-3541d380-eae6-4cd1-b27c-e2b06a8d025e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "53f34c67-563c-4a95-ac23-25e7d05d305d", "name": "network-vif-deleted", "tag": "c60dc47c-8f9b-48bf-b642-4c2f82a01fe6"}]} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:48:03.800443 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-d188caa3-1014-4f5a-821b-46749ea223af req-3541d380-eae6-4cd1-b27c-e2b06a8d025e service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:03.800927 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [req-d188caa3-1014-4f5a-821b-46749ea223af req-3541d380-eae6-4cd1-b27c-e2b06a8d025e service nova] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:03.812227 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.compute.server_external_events [req-d188caa3-1014-4f5a-821b-46749ea223af req-3541d380-eae6-4cd1-b27c-e2b06a8d025e service nova] Creating event network-vif-deleted:c60dc47c-8f9b-48bf-b642-4c2f82a01fe6 for instance 53f34c67-563c-4a95-ac23-25e7d05d305d on hv-cinder-maste Jun 10 13:48:03.817225 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [req-d188caa3-1014-4f5a-821b-46749ea223af req-3541d380-eae6-4cd1-b27c-e2b06a8d025e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028887 Jun 10 13:48:03.817866 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1726/2640] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Jun 10 13:48:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:48:03.822792 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-2e3fe0a8-5995-416c-92ec-0be47a03b623 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:03.823471 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-2e3fe0a8-5995-416c-92ec-0be47a03b623 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:48:03.830384 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e3fe0a8-5995-416c-92ec-0be47a03b623 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:03.830831 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e3fe0a8-5995-416c-92ec-0be47a03b623 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:03.831576 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e3fe0a8-5995-416c-92ec-0be47a03b623 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:03.832008 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-2e3fe0a8-5995-416c-92ec-0be47a03b623 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:03.895431 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-2e3fe0a8-5995-416c-92ec-0be47a03b623 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:48:03.902939 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-2e3fe0a8-5995-416c-92ec-0be47a03b623 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1390 microversion: 2.1 time: 0.083991 Jun 10 13:48:03.903501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1727/2641] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:48:03 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1390 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jun 10 13:48:04.415092 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:48:04.415294 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:48:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:48:04.919465 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3868d2d0-fc93-483a-b126-276a527e3855 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:04.920088 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-3868d2d0-fc93-483a-b126-276a527e3855 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:48:04.926510 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3868d2d0-fc93-483a-b126-276a527e3855 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:04.926957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3868d2d0-fc93-483a-b126-276a527e3855 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:04.927713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3868d2d0-fc93-483a-b126-276a527e3855 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:04.928151 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3868d2d0-fc93-483a-b126-276a527e3855 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:04.958643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3868d2d0-fc93-483a-b126-276a527e3855 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:04.959090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-3868d2d0-fc93-483a-b126-276a527e3855 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:04.959530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.objects.instance [None req-3868d2d0-fc93-483a-b126-276a527e3855 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lazy-loading 'fault' on Instance uuid 53f34c67-563c-4a95-ac23-25e7d05d305d {{(pid=118092) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jun 10 13:48:04.998259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.policy [None req-3868d2d0-fc93-483a-b126-276a527e3855 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bf961149e78d4c0c8acb58f1cec447bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '372f8b4d0d5c4046ad91bea6d709a6ef', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118092) authorize /opt/stack/nova/nova/policy.py:199}} Jun 10 13:48:05.005767 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3868d2d0-fc93-483a-b126-276a527e3855 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 200 len: 1230 microversion: 2.1 time: 0.090224 Jun 10 13:48:05.009730 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1728/2642] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:48:04 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 1230 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jun 10 13:48:06.023259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e35a8806-834f-4045-a217-d9aa2581c37e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:06.023862 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-e35a8806-834f-4045-a217-d9aa2581c37e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 53f34c67-563c-4a95-ac23-25e7d05d305d] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:48:06.030769 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e35a8806-834f-4045-a217-d9aa2581c37e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:06.031212 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e35a8806-834f-4045-a217-d9aa2581c37e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:06.031995 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e35a8806-834f-4045-a217-d9aa2581c37e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:06.032427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-e35a8806-834f-4045-a217-d9aa2581c37e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:06.045815 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-e35a8806-834f-4045-a217-d9aa2581c37e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] HTTP exception thrown: Instance 53f34c67-563c-4a95-ac23-25e7d05d305d could not be found. Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e35a8806-834f-4045-a217-d9aa2581c37e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Request method failure captured: Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d HTTP/1.1 Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0naMDwEZ-YRSB7u9MWmuxa7VCZvbw5ekhxDtysZCwuiejBDwGfJK6HGQuuDfJjuNsnxxtAcrbNyuDjyZEn2ArGkghDHQNGqM2deKRpOfTSFnRaPrcSQSdBg7vhdrLqh7OJInyc42VEgwet4IzwTw2sa1gRaoreRqelNgRh5zNivNGE Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 372f8b4d0d5c4046ad91bea6d709a6ef Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:06.047410 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:06.048493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_372f8b4d0d5c4046ad91bea6d709a6ef", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:06.048493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:48:06.048493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 372f8b4d0d5c4046ad91bea6d709a6ef Jun 10 13:48:06.048493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:48:06.048493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-TestVolumeBootPattern-1295034315-project Jun 10 13:48:06.048493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:06.048493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:06.048493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: bf961149e78d4c0c8acb58f1cec447bd Jun 10 13:48:06.048493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-TestVolumeBootPattern-1295034315-project Jun 10 13:48:06.048493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:48:06.048493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 53f34c67-563c-4a95-ac23-25e7d05d305d could not be found. Jun 10 13:48:06.048493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '53f34c67-563c-4a95-ac23-25e7d05d305d'} Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 53f34c67-563c-4a95-ac23-25e7d05d305d could not be found. Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:06.049162 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 53f34c67-563c-4a95-ac23-25e7d05d305d could not be found. Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 53f34c67-563c-4a95-ac23-25e7d05d305d could not be found. Jun 10 13:48:06.050254 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.051412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-e35a8806-834f-4045-a217-d9aa2581c37e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Returning 404 to user: Instance 53f34c67-563c-4a95-ac23-25e7d05d305d could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:06.051412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-e35a8806-834f-4045-a217-d9aa2581c37e tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d" status: 404 len: 111 microversion: 2.1 time: 0.029449 Jun 10 13:48:06.051412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1729/2643] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:48:06 2022] GET /compute/v2.1/servers/53f34c67-563c-4a95-ac23-25e7d05d305d => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:48:06.545518 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0d490fc8-0e2d-40d6-aa88-01cda9fde8ce tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:06.546015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-0d490fc8-0e2d-40d6-aa88-01cda9fde8ce tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:48:06.552042 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0d490fc8-0e2d-40d6-aa88-01cda9fde8ce tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:06.556594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0d490fc8-0e2d-40d6-aa88-01cda9fde8ce tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:06.556594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0d490fc8-0e2d-40d6-aa88-01cda9fde8ce tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:06.556594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-0d490fc8-0e2d-40d6-aa88-01cda9fde8ce tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:06.566913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-0d490fc8-0e2d-40d6-aa88-01cda9fde8ce tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] HTTP exception thrown: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0d490fc8-0e2d-40d6-aa88-01cda9fde8ce tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Request method failure captured: Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 HTTP/1.1 Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0naMDwEZ-YRSB7u9MWmuxa7VCZvbw5ekhxDtysZCwuiejBDwGfJK6HGQuuDfJjuNsnxxtAcrbNyuDjyZEn2ArGkghDHQNGqM2deKRpOfTSFnRaPrcSQSdBg7vhdrLqh7OJInyc42VEgwet4IzwTw2sa1gRaoreRqelNgRh5zNivNGE Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 372f8b4d0d5c4046ad91bea6d709a6ef Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:06.568430 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:06.569573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_372f8b4d0d5c4046ad91bea6d709a6ef", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:06.569573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:48:06.569573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 372f8b4d0d5c4046ad91bea6d709a6ef Jun 10 13:48:06.569573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:48:06.569573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-TestVolumeBootPattern-1295034315-project Jun 10 13:48:06.569573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:06.569573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:06.569573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: bf961149e78d4c0c8acb58f1cec447bd Jun 10 13:48:06.569573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-TestVolumeBootPattern-1295034315-project Jun 10 13:48:06.569573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:48:06.569573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:48:06.569573 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '8b33753e-2717-477d-b8f7-1f636fee10c4'} Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:06.570200 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:48:06.571382 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.572643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0d490fc8-0e2d-40d6-aa88-01cda9fde8ce tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Returning 404 to user: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:06.572643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0d490fc8-0e2d-40d6-aa88-01cda9fde8ce tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "DELETE /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 404 len: 111 microversion: 2.1 time: 0.028281 Jun 10 13:48:06.572643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1730/2644] 10.222.0.25 () {58 vars in 1238 bytes} [Fri Jun 10 13:48:06 2022] DELETE /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:48:06.578624 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-480d58f7-00cd-4dba-a64d-0b8a4ff22d86 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:06.579220 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-480d58f7-00cd-4dba-a64d-0b8a4ff22d86 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] [instance: 8b33753e-2717-477d-b8f7-1f636fee10c4] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:48:06.584899 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-480d58f7-00cd-4dba-a64d-0b8a4ff22d86 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:06.585353 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-480d58f7-00cd-4dba-a64d-0b8a4ff22d86 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:06.586143 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-480d58f7-00cd-4dba-a64d-0b8a4ff22d86 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:06.586530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-480d58f7-00cd-4dba-a64d-0b8a4ff22d86 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:06.590407 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:48:06.600015 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-480d58f7-00cd-4dba-a64d-0b8a4ff22d86 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] HTTP exception thrown: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-480d58f7-00cd-4dba-a64d-0b8a4ff22d86 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Request method failure captured: Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 HTTP/1.1 Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0naMDwEZ-YRSB7u9MWmuxa7VCZvbw5ekhxDtysZCwuiejBDwGfJK6HGQuuDfJjuNsnxxtAcrbNyuDjyZEn2ArGkghDHQNGqM2deKRpOfTSFnRaPrcSQSdBg7vhdrLqh7OJInyc42VEgwet4IzwTw2sa1gRaoreRqelNgRh5zNivNGE Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 372f8b4d0d5c4046ad91bea6d709a6ef Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:06.601523 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:06.602796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_372f8b4d0d5c4046ad91bea6d709a6ef", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/372f8b4d0d5c4046ad91bea6d709a6ef"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:06.602796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:48:06.602796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 372f8b4d0d5c4046ad91bea6d709a6ef Jun 10 13:48:06.602796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-TestVolumeBootPattern-1295034315 Jun 10 13:48:06.602796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-TestVolumeBootPattern-1295034315-project Jun 10 13:48:06.602796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:06.602796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:06.602796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: bf961149e78d4c0c8acb58f1cec447bd Jun 10 13:48:06.602796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-TestVolumeBootPattern-1295034315-project Jun 10 13:48:06.602796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:48:06.602796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:48:06.602796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '8b33753e-2717-477d-b8f7-1f636fee10c4'} Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:06.603464 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. Jun 10 13:48:06.604818 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:06.609291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-480d58f7-00cd-4dba-a64d-0b8a4ff22d86 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Returning 404 to user: Instance 8b33753e-2717-477d-b8f7-1f636fee10c4 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:06.609291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-480d58f7-00cd-4dba-a64d-0b8a4ff22d86 tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4" status: 404 len: 111 microversion: 2.1 time: 0.028661 Jun 10 13:48:06.609291 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1731/2645] 10.222.0.25 () {58 vars in 1235 bytes} [Fri Jun 10 13:48:06 2022] GET /compute/v2.1/servers/8b33753e-2717-477d-b8f7-1f636fee10c4 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:48:08.100959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-31d8d90b-6cc5-4912-a43d-078f36a5645a tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] Calling method '.version_select at 0x7f9305218790>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:08.177925 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-31d8d90b-6cc5-4912-a43d-078f36a5645a tempest-TestVolumeBootPattern-1295034315 tempest-TestVolumeBootPattern-1295034315-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-700124936" status: 202 len: 0 microversion: 2.1 time: 0.086159 Jun 10 13:48:08.178511 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1732/2646] 10.222.0.25 () {58 vars in 1259 bytes} [Fri Jun 10 13:48:08 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-700124936 => generated 0 bytes in 87 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:48:22.567364 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=53,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:48:22.571293 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-865c494e-8dcb-44fa-83e7-3bbf787f1042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:22.571808 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-865c494e-8dcb-44fa-83e7-3bbf787f1042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:48:22.579725 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-865c494e-8dcb-44fa-83e7-3bbf787f1042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:22.580169 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-865c494e-8dcb-44fa-83e7-3bbf787f1042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:22.580955 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-865c494e-8dcb-44fa-83e7-3bbf787f1042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:22.581387 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-865c494e-8dcb-44fa-83e7-3bbf787f1042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:22.593913 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-865c494e-8dcb-44fa-83e7-3bbf787f1042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] HTTP exception thrown: Instance db56898a-dff9-4cad-bf8d-f84b53247fc3 could not be found. Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-865c494e-8dcb-44fa-83e7-3bbf787f1042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Request method failure captured: Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 HTTP/1.1 Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0GI4neGqGl6u4NtxR88mPQrCvR_v1VmiIEaXPKEqlwdyaT_vIeeEJpWEM0993X6W9IucjryiBRG6JJNHEx9ulNG1lmVs69VgCwa572eOjYLN6_EoRIIXPkdDj4Zac85isYH3jNfGblme5mU9zzCrvjMFkL5ny73zNQIxIWZatGPexU Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Openstack-Nova-Api-Version: 2.20 Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 28f8244decff453e98e2d372a97a563c Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:22.595379 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:22.596857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/28f8244decff453e98e2d372a97a563c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/28f8244decff453e98e2d372a97a563c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_28f8244decff453e98e2d372a97a563c", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/28f8244decff453e98e2d372a97a563c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:22.596857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:48:22.596857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 28f8244decff453e98e2d372a97a563c Jun 10 13:48:22.596857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:48:22.596857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeShelveTestJSON-1465745724-project Jun 10 13:48:22.596857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:22.596857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:22.596857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: b1d1712ea523498f92f258383863b906 Jun 10 13:48:22.596857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1465745724-project Jun 10 13:48:22.596857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:48:22.596857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance db56898a-dff9-4cad-bf8d-f84b53247fc3 could not be found. Jun 10 13:48:22.596857 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'db56898a-dff9-4cad-bf8d-f84b53247fc3'} Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance db56898a-dff9-4cad-bf8d-f84b53247fc3 could not be found. Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:22.597643 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance db56898a-dff9-4cad-bf8d-f84b53247fc3 could not be found. Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance db56898a-dff9-4cad-bf8d-f84b53247fc3 could not be found. Jun 10 13:48:22.599150 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:22.600665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-865c494e-8dcb-44fa-83e7-3bbf787f1042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Returning 404 to user: Instance db56898a-dff9-4cad-bf8d-f84b53247fc3 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:22.600665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-865c494e-8dcb-44fa-83e7-3bbf787f1042 tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "DELETE /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 404 len: 111 microversion: 2.20 time: 0.030957 Jun 10 13:48:22.600665 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1733/2647] 10.222.0.25 () {60 vars in 1279 bytes} [Fri Jun 10 13:48:22 2022] DELETE /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jun 10 13:48:22.604335 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-91325a3f-8f1d-4586-8b4f-5445292a00cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Calling method '>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:22.604959 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.compute.api [None req-91325a3f-8f1d-4586-8b4f-5445292a00cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] [instance: db56898a-dff9-4cad-bf8d-f84b53247fc3] Fetching instance by UUID {{(pid=118092) get /opt/stack/nova/nova/compute/api.py:2920}} Jun 10 13:48:22.610296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91325a3f-8f1d-4586-8b4f-5445292a00cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:22.610737 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91325a3f-8f1d-4586-8b4f-5445292a00cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:22.611469 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91325a3f-8f1d-4586-8b4f-5445292a00cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jun 10 13:48:22.611928 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG oslo_concurrency.lockutils [None req-91325a3f-8f1d-4586-8b4f-5445292a00cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Lock "bf155fd7-c737-40a3-a522-fd4049b0e8da" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118092) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jun 10 13:48:22.625184 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-91325a3f-8f1d-4586-8b4f-5445292a00cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] HTTP exception thrown: Instance db56898a-dff9-4cad-bf8d-f84b53247fc3 could not be found. Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-91325a3f-8f1d-4586-8b4f-5445292a00cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Request method failure captured: Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 HTTP/1.1 Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0GI4neGqGl6u4NtxR88mPQrCvR_v1VmiIEaXPKEqlwdyaT_vIeeEJpWEM0993X6W9IucjryiBRG6JJNHEx9ulNG1lmVs69VgCwa572eOjYLN6_EoRIIXPkdDj4Zac85isYH3jNfGblme5mU9zzCrvjMFkL5ny73zNQIxIWZatGPexU Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Openstack-Nova-Api-Version: 2.20 Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 28f8244decff453e98e2d372a97a563c Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:22.626647 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:22.628090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/28f8244decff453e98e2d372a97a563c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/28f8244decff453e98e2d372a97a563c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_28f8244decff453e98e2d372a97a563c", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/28f8244decff453e98e2d372a97a563c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:22.628090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:48:22.628090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 28f8244decff453e98e2d372a97a563c Jun 10 13:48:22.628090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1465745724 Jun 10 13:48:22.628090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-AttachVolumeShelveTestJSON-1465745724-project Jun 10 13:48:22.628090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:22.628090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:22.628090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: b1d1712ea523498f92f258383863b906 Jun 10 13:48:22.628090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1465745724-project Jun 10 13:48:22.628090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: > Jun 10 13:48:22.628090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Instance db56898a-dff9-4cad-bf8d-f84b53247fc3 could not be found. Jun 10 13:48:22.628090 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'db56898a-dff9-4cad-bf8d-f84b53247fc3'} Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance db56898a-dff9-4cad-bf8d-f84b53247fc3 could not be found. Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:22.634299 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance db56898a-dff9-4cad-bf8d-f84b53247fc3 could not be found. Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance db56898a-dff9-4cad-bf8d-f84b53247fc3 could not be found. Jun 10 13:48:22.635568 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:22.636687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-91325a3f-8f1d-4586-8b4f-5445292a00cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] Returning 404 to user: Instance db56898a-dff9-4cad-bf8d-f84b53247fc3 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:22.636687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-91325a3f-8f1d-4586-8b4f-5445292a00cb tempest-AttachVolumeShelveTestJSON-1465745724 tempest-AttachVolumeShelveTestJSON-1465745724-project] 10.222.0.25 "GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3" status: 404 len: 111 microversion: 2.20 time: 0.027009 Jun 10 13:48:22.636687 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1734/2648] 10.222.0.25 () {60 vars in 1276 bytes} [Fri Jun 10 13:48:22 2022] GET /compute/v2.1/servers/db56898a-dff9-4cad-bf8d-f84b53247fc3 => generated 111 bytes in 28 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jun 10 13:48:28.434488 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-ff6d8d37-4b80-49d0-a8d4-17f3ab9049d0 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Action: 'create', calling method: .version_select at 0x7f9309921790>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-766704396"}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:48:29.011059 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-ff6d8d37-4b80-49d0-a8d4-17f3ab9049d0 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 0.687320 Jun 10 13:48:29.011690 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1735/2649] 10.222.0.25 () {60 vars in 1154 bytes} [Fri Jun 10 13:48:28 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 689 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Jun 10 13:48:29.022096 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-331e4c95-536d-44cc-96dd-0f5fa88bd35d tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f93099fb0d0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:29.070000 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-331e4c95-536d-44cc-96dd-0f5fa88bd35d tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189" status: 200 len: 344 microversion: 2.1 time: 0.051482 Jun 10 13:48:29.070608 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1736/2650] 10.222.0.25 () {58 vars in 1244 bytes} [Fri Jun 10 13:48:29 2022] GET /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189 => generated 344 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:48:30.082907 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-7fca0010-40ae-4789-8188-b0ffa0f1f39d tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f9309a7c1f0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:30.128935 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7fca0010-40ae-4789-8188-b0ffa0f1f39d tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189" status: 200 len: 345 microversion: 2.1 time: 0.050265 Jun 10 13:48:30.129501 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1737/2651] 10.222.0.25 () {58 vars in 1244 bytes} [Fri Jun 10 13:48:30 2022] GET /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189 => generated 345 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:48:30.139850 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-9e8b2b94-235e-4669-a643-25444d7c3b3b tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Action: 'create', calling method: .version_select at 0x7f9309921dc0>, body: {"snapshot": {"volume_id": "7e8cd94b-b313-4eae-919c-f002c17d5189", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1683630694"}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:48:30.596783 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-9e8b2b94-235e-4669-a643-25444d7c3b3b tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.460347 Jun 10 13:48:30.597427 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1738/2652] 10.222.0.25 () {60 vars in 1161 bytes} [Fri Jun 10 13:48:30 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:48:30.607194 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-00bcce41-d936-4767-9303-72f85048db12 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f93054afee0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:30.636020 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-00bcce41-d936-4767-9303-72f85048db12 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f" status: 200 len: 287 microversion: 2.1 time: 0.032722 Jun 10 13:48:30.636616 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1739/2653] 10.222.0.25 () {58 vars in 1250 bytes} [Fri Jun 10 13:48:30 2022] GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f => generated 287 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:48:30.646232 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-c417ebb6-60de-4f42-a53c-48803bba37fb tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f93099fb1f0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:30.675173 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-c417ebb6-60de-4f42-a53c-48803bba37fb tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.032339 Jun 10 13:48:30.675759 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1740/2654] 10.222.0.25 () {58 vars in 1139 bytes} [Fri Jun 10 13:48:30 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:48:30.685074 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-df999706-6c58-47ef-b482-a6938113ef33 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f9305341d30>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:30.717625 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-df999706-6c58-47ef-b482-a6938113ef33 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f" status: 200 len: 287 microversion: 2.1 time: 0.035772 Jun 10 13:48:30.718320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1741/2655] 10.222.0.25 () {58 vars in 1250 bytes} [Fri Jun 10 13:48:30 2022] GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f => generated 287 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:48:31.729634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-295e788b-360f-40d6-9e61-2c23ab6beb36 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f93099219d0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:31.758747 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-295e788b-360f-40d6-9e61-2c23ab6beb36 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f" status: 200 len: 287 microversion: 2.1 time: 0.033315 Jun 10 13:48:31.759314 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1742/2656] 10.222.0.25 () {58 vars in 1250 bytes} [Fri Jun 10 13:48:31 2022] GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f => generated 287 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:48:32.767859 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_cell1:SELECT=4 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:48:32.772187 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-3af24328-7643-4b42-bdfb-09dc23a98e59 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f93053398b0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:32.802176 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-3af24328-7643-4b42-bdfb-09dc23a98e59 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f" status: 200 len: 287 microversion: 2.1 time: 0.035072 Jun 10 13:48:32.802736 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1743/2657] 10.222.0.25 () {58 vars in 1250 bytes} [Fri Jun 10 13:48:32 2022] GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f => generated 287 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:48:33.814733 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-cb134789-9905-4fb4-9623-5cb466eaba6d tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f9307095b80>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:33.843434 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-cb134789-9905-4fb4-9623-5cb466eaba6d tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f" status: 200 len: 287 microversion: 2.1 time: 0.033464 Jun 10 13:48:33.843978 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1744/2658] 10.222.0.25 () {58 vars in 1250 bytes} [Fri Jun 10 13:48:33 2022] GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f => generated 287 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:48:34.856737 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-71ec7163-e836-4281-8fd5-c2b7724e55a0 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f9305341dc0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:34.887748 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-71ec7163-e836-4281-8fd5-c2b7724e55a0 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f" status: 200 len: 288 microversion: 2.1 time: 0.035433 Jun 10 13:48:34.888341 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1745/2659] 10.222.0.25 () {58 vars in 1250 bytes} [Fri Jun 10 13:48:34 2022] GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f => generated 288 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:48:34.898806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-93c82a77-af17-45fc-85bc-f4e3c319d9c8 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f9309a7c820>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:35.012030 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-93c82a77-af17-45fc-85bc-f4e3c319d9c8 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "DELETE /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f" status: 202 len: 0 microversion: 2.1 time: 0.116619 Jun 10 13:48:35.012601 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1746/2660] 10.222.0.25 () {58 vars in 1253 bytes} [Fri Jun 10 13:48:34 2022] DELETE /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f => generated 0 bytes in 118 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:48:35.020989 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-196b81cd-8b11-40c0-9da1-89c9dd7ee4a8 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f9309921c10>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:35.050155 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-196b81cd-8b11-40c0-9da1-89c9dd7ee4a8 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f" status: 200 len: 287 microversion: 2.1 time: 0.032769 Jun 10 13:48:35.050714 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1747/2661] 10.222.0.25 () {58 vars in 1250 bytes} [Fri Jun 10 13:48:35 2022] GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f => generated 287 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:48:36.062117 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1e256f38-6ce5-4c9f-989c-e8bb660eeb4b tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f93052184c0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:36.086071 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-1e256f38-6ce5-4c9f-989c-e8bb660eeb4b tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] HTTP exception thrown: Snapshot c6463e4b-37f6-4343-b5ef-29c17abf763f could not be found. Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1e256f38-6ce5-4c9f-989c-e8bb660eeb4b tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Request method failure captured: Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f HTTP/1.1 Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0ussWnbHwFw_9yHV_QWQgIkJoPOl0AnLXlN63ZXqy1a9U_Lv7J3ITzYXY3x5PboE7jkjvTepY-tLvsxLA3Q8Fdz_LnbjGPC0GcsURcByc51KY6PMxQn5MBYrT6ORZL2OHI6UVNg98o0rKido5Jt_IOicOHcYDrBaOo52D1g_Q1tuA8 Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 712acea883074881abb993d97efe6fb2 Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1579861680 Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:36.088530 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:36.089957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/712acea883074881abb993d97efe6fb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/712acea883074881abb993d97efe6fb2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_712acea883074881abb993d97efe6fb2", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/712acea883074881abb993d97efe6fb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:36.089957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1579861680 Jun 10 13:48:36.089957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 712acea883074881abb993d97efe6fb2 Jun 10 13:48:36.089957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1579861680 Jun 10 13:48:36.089957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesSnapshotsTestJSON-1579861680-project Jun 10 13:48:36.089957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:36.089957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:36.089957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 389f5193a266449a8181dd2e86b69f1c Jun 10 13:48:36.089957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1579861680-project Jun 10 13:48:36.089957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: Jun 10 13:48:36.089957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Snapshot c6463e4b-37f6-4343-b5ef-29c17abf763f could not be found. Jun 10 13:48:36.089957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'c6463e4b-37f6-4343-b5ef-29c17abf763f'} Jun 10 13:48:36.089957 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot c6463e4b-37f6-4343-b5ef-29c17abf763f could not be found. (HTTP 404) (Request-ID: req-d1465bd9-ad9c-45e4-b84f-7aaca6887d8f) Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:36.090761 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jun 10 13:48:36.091826 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot c6463e4b-37f6-4343-b5ef-29c17abf763f could not be found. Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Jun 10 13:48:36.094821 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:48:36.096342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot c6463e4b-37f6-4343-b5ef-29c17abf763f could not be found. Jun 10 13:48:36.096342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:36.096342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:36.096342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:36.096342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:36.096342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:36.096342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:36.096342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:48:36.096342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:48:36.096342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot c6463e4b-37f6-4343-b5ef-29c17abf763f could not be found. Jun 10 13:48:36.096342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:36.096342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-1e256f38-6ce5-4c9f-989c-e8bb660eeb4b tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Returning 404 to user: Snapshot c6463e4b-37f6-4343-b5ef-29c17abf763f could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:36.096342 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-1e256f38-6ce5-4c9f-989c-e8bb660eeb4b tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f" status: 404 len: 111 microversion: 2.1 time: 0.032227 Jun 10 13:48:36.097309 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1748/2662] 10.222.0.25 () {58 vars in 1250 bytes} [Fri Jun 10 13:48:36 2022] GET /compute/v2.1/os-snapshots/c6463e4b-37f6-4343-b5ef-29c17abf763f => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:48:36.103577 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d709c187-3675-4c8f-b154-080ae12c61ab tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f9306f35790>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:36.279659 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d709c187-3675-4c8f-b154-080ae12c61ab tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189" status: 202 len: 0 microversion: 2.1 time: 0.184153 Jun 10 13:48:36.280951 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1749/2663] 10.222.0.25 () {58 vars in 1247 bytes} [Fri Jun 10 13:48:36 2022] DELETE /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189 => generated 0 bytes in 186 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jun 10 13:48:36.288731 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d0943d3f-4bf2-484a-8dca-c012c162dee4 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f9309921d30>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:36.337086 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d0943d3f-4bf2-484a-8dca-c012c162dee4 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189" status: 200 len: 344 microversion: 2.1 time: 0.051911 Jun 10 13:48:36.337658 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1750/2664] 10.222.0.25 () {58 vars in 1244 bytes} [Fri Jun 10 13:48:36 2022] GET /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189 => generated 344 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jun 10 13:48:37.350127 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0eb23733-00c3-4dd6-9ce9-17c891585259 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f9305554c10>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:37.378051 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-0eb23733-00c3-4dd6-9ce9-17c891585259 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] HTTP exception thrown: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0eb23733-00c3-4dd6-9ce9-17c891585259 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Request method failure captured: Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189 HTTP/1.1 Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0ussWnbHwFw_9yHV_QWQgIkJoPOl0AnLXlN63ZXqy1a9U_Lv7J3ITzYXY3x5PboE7jkjvTepY-tLvsxLA3Q8Fdz_LnbjGPC0GcsURcByc51KY6PMxQn5MBYrT6ORZL2OHI6UVNg98o0rKido5Jt_IOicOHcYDrBaOo52D1g_Q1tuA8 Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 712acea883074881abb993d97efe6fb2 Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1579861680 Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:37.379999 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:37.381325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/712acea883074881abb993d97efe6fb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/712acea883074881abb993d97efe6fb2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_712acea883074881abb993d97efe6fb2", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/712acea883074881abb993d97efe6fb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:37.381325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1579861680 Jun 10 13:48:37.381325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 712acea883074881abb993d97efe6fb2 Jun 10 13:48:37.381325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1579861680 Jun 10 13:48:37.381325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesSnapshotsTestJSON-1579861680-project Jun 10 13:48:37.381325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:37.381325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:37.381325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 389f5193a266449a8181dd2e86b69f1c Jun 10 13:48:37.381325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1579861680-project Jun 10 13:48:37.381325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: Jun 10 13:48:37.381325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.381325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '7e8cd94b-b313-4eae-919c-f002c17d5189'} Jun 10 13:48:37.381325 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi item = cinderclient( Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. (HTTP 404) (Request-ID: req-b57dbef5-1837-4531-86be-c6f8489ee036) Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:37.382022 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi item = cinderclient( Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jun 10 13:48:37.383484 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jun 10 13:48:37.384753 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:48:37.386230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.386230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.386230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:37.386230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.386230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:37.386230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:37.386230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:37.386230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:48:37.386230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:48:37.386230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.386230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.386230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0eb23733-00c3-4dd6-9ce9-17c891585259 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Returning 404 to user: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:37.386230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0eb23733-00c3-4dd6-9ce9-17c891585259 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189" status: 404 len: 109 microversion: 2.1 time: 0.035730 Jun 10 13:48:37.386230 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1751/2665] 10.222.0.25 () {58 vars in 1244 bytes} [Fri Jun 10 13:48:37 2022] GET /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:48:37.392320 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d6eeab04-8633-44ef-90ea-4762bbc1bf88 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f93099fbf70>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:37.421239 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-d6eeab04-8633-44ef-90ea-4762bbc1bf88 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] HTTP exception thrown: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d6eeab04-8633-44ef-90ea-4762bbc1bf88 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Request method failure captured: Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189 HTTP/1.1 Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0ussWnbHwFw_9yHV_QWQgIkJoPOl0AnLXlN63ZXqy1a9U_Lv7J3ITzYXY3x5PboE7jkjvTepY-tLvsxLA3Q8Fdz_LnbjGPC0GcsURcByc51KY6PMxQn5MBYrT6ORZL2OHI6UVNg98o0rKido5Jt_IOicOHcYDrBaOo52D1g_Q1tuA8 Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 712acea883074881abb993d97efe6fb2 Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1579861680 Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:37.423188 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:37.424372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/712acea883074881abb993d97efe6fb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/712acea883074881abb993d97efe6fb2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_712acea883074881abb993d97efe6fb2", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/712acea883074881abb993d97efe6fb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:37.424372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1579861680 Jun 10 13:48:37.424372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 712acea883074881abb993d97efe6fb2 Jun 10 13:48:37.424372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1579861680 Jun 10 13:48:37.424372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesSnapshotsTestJSON-1579861680-project Jun 10 13:48:37.424372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:37.424372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:37.424372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 389f5193a266449a8181dd2e86b69f1c Jun 10 13:48:37.424372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1579861680-project Jun 10 13:48:37.424372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: Jun 10 13:48:37.424372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.424372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '7e8cd94b-b313-4eae-919c-f002c17d5189'} Jun 10 13:48:37.424372 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. (HTTP 404) (Request-ID: req-dc1c240b-2bfb-4173-a2bd-c096be6cd762) Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:37.425107 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jun 10 13:48:37.426168 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jun 10 13:48:37.427478 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:48:37.428676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.428676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.428676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:37.428676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.428676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:37.428676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:37.428676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:37.428676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:48:37.428676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:48:37.428676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.428676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.428676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d6eeab04-8633-44ef-90ea-4762bbc1bf88 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Returning 404 to user: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:37.428676 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d6eeab04-8633-44ef-90ea-4762bbc1bf88 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189" status: 404 len: 109 microversion: 2.1 time: 0.036696 Jun 10 13:48:37.429524 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1752/2666] 10.222.0.25 () {58 vars in 1247 bytes} [Fri Jun 10 13:48:37 2022] DELETE /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189 => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:48:37.432700 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d5c82de1-24ba-496f-973f-15736f2ff4b0 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Calling method '.version_select at 0x7f9309921d30>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:37.459288 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-d5c82de1-24ba-496f-973f-15736f2ff4b0 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] HTTP exception thrown: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d5c82de1-24ba-496f-973f-15736f2ff4b0 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Request method failure captured: Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189 HTTP/1.1 Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0ussWnbHwFw_9yHV_QWQgIkJoPOl0AnLXlN63ZXqy1a9U_Lv7J3ITzYXY3x5PboE7jkjvTepY-tLvsxLA3Q8Fdz_LnbjGPC0GcsURcByc51KY6PMxQn5MBYrT6ORZL2OHI6UVNg98o0rKido5Jt_IOicOHcYDrBaOo52D1g_Q1tuA8 Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: 712acea883074881abb993d97efe6fb2 Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1579861680 Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:37.461245 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:37.462634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/712acea883074881abb993d97efe6fb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/712acea883074881abb993d97efe6fb2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_712acea883074881abb993d97efe6fb2", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/712acea883074881abb993d97efe6fb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:37.462634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1579861680 Jun 10 13:48:37.462634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: 712acea883074881abb993d97efe6fb2 Jun 10 13:48:37.462634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1579861680 Jun 10 13:48:37.462634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesSnapshotsTestJSON-1579861680-project Jun 10 13:48:37.462634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:37.462634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:37.462634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 389f5193a266449a8181dd2e86b69f1c Jun 10 13:48:37.462634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1579861680-project Jun 10 13:48:37.462634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: Jun 10 13:48:37.462634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.462634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '7e8cd94b-b313-4eae-919c-f002c17d5189'} Jun 10 13:48:37.462634 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi item = cinderclient( Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. (HTTP 404) (Request-ID: req-ac868fc8-485c-4370-ae00-2e0f569c0ab9) Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:37.463493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi item = cinderclient( Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jun 10 13:48:37.464830 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jun 10 13:48:37.471277 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:48:37.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:37.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:37.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:37.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:37.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:48:37.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:48:37.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. Jun 10 13:48:37.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:37.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-d5c82de1-24ba-496f-973f-15736f2ff4b0 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] Returning 404 to user: Volume 7e8cd94b-b313-4eae-919c-f002c17d5189 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:37.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-d5c82de1-24ba-496f-973f-15736f2ff4b0 tempest-VolumesSnapshotsTestJSON-1579861680 tempest-VolumesSnapshotsTestJSON-1579861680-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189" status: 404 len: 109 microversion: 2.1 time: 0.033551 Jun 10 13:48:37.472892 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1753/2667] 10.222.0.25 () {58 vars in 1244 bytes} [Fri Jun 10 13:48:37 2022] GET /compute/v2.1/os-volumes/7e8cd94b-b313-4eae-919c-f002c17d5189 => generated 109 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:48:39.346705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG dbcounter [-] [118092] Writing DB stats nova_api:SELECT=4 {{(pid=118092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:48:39.456129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0ef2bcb0-0b4a-4f52-8937-6efaa99f36de tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Action: 'create', calling method: .version_select at 0x7f9305483f70>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-905332940", "metadata": {"Type": "work"}}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0ef2bcb0-0b4a-4f52-8937-6efaa99f36de tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Request method failure captured: Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 121 Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0u3JYkGwHkg4Rvh8qshGIAEBVyvEoRVNULJfdTIofcDwVy5b95KZtTtxufNN-I6YkJrYLJy0TK2puGHPkh9PG0l8m0Zs6moScv-yjXK0vdyc2oRt3sC3_sd-YXc1kWX8-f8AP4tSB_1vVfbybGqGatpfc59wUj2ffRdMvYQNtu1gsw Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: c8b90db7b2fa44269a94c63ea2512159 Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:39.483412 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:39.484796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_c8b90db7b2fa44269a94c63ea2512159", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:39.484796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.484796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: c8b90db7b2fa44269a94c63ea2512159 Jun 10 13:48:39.484796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.484796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesNegativeTest-1467618453-project Jun 10 13:48:39.484796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:39.484796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:39.484796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 9af12e2d68674326a7f217252b583a98 Jun 10 13:48:39.484796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesNegativeTest-1467618453-project Jun 10 13:48:39.484796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Jun 10 13:48:39.484796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-905332940", "metadata": {"Type": "work"}}} Jun 10 13:48:39.484796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: Jun 10 13:48:39.484796 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-905332940', 'metadata': {'Type': 'work'}}}} Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise error Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi On instance['volume']: Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi '#$%' Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:39.485627 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jun 10 13:48:39.486861 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jun 10 13:48:39.488082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jun 10 13:48:39.488082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.488082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-0ef2bcb0-0b4a-4f52-8937-6efaa99f36de tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:39.488082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-0ef2bcb0-0b4a-4f52-8937-6efaa99f36de tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.139927 Jun 10 13:48:39.488082 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1754/2668] 10.222.0.25 () {60 vars in 1155 bytes} [Fri Jun 10 13:48:39 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 141 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jun 10 13:48:39.493795 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dbf3596c-fa5f-4cc1-a0ce-6d20a221fc47 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Action: 'create', calling method: .version_select at 0x7f9305339d30>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-384085296", "metadata": {"Type": "work"}}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dbf3596c-fa5f-4cc1-a0ce-6d20a221fc47 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Request method failure captured: Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 119 Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0u3JYkGwHkg4Rvh8qshGIAEBVyvEoRVNULJfdTIofcDwVy5b95KZtTtxufNN-I6YkJrYLJy0TK2puGHPkh9PG0l8m0Zs6moScv-yjXK0vdyc2oRt3sC3_sd-YXc1kWX8-f8AP4tSB_1vVfbybGqGatpfc59wUj2ffRdMvYQNtu1gsw Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: c8b90db7b2fa44269a94c63ea2512159 Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:39.495981 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:39.497374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_c8b90db7b2fa44269a94c63ea2512159", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:39.497374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.497374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: c8b90db7b2fa44269a94c63ea2512159 Jun 10 13:48:39.497374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.497374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesNegativeTest-1467618453-project Jun 10 13:48:39.497374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:39.497374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:39.497374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 9af12e2d68674326a7f217252b583a98 Jun 10 13:48:39.497374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesNegativeTest-1467618453-project Jun 10 13:48:39.497374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Jun 10 13:48:39.497374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-384085296", "metadata": {"Type": "work"}}} Jun 10 13:48:39.497374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: Jun 10 13:48:39.497374 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-384085296', 'metadata': {'Type': 'work'}}}} Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise error Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi On instance['volume']: Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi '0' Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:39.498135 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jun 10 13:48:39.499267 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jun 10 13:48:39.500221 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jun 10 13:48:39.500221 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.500221 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-dbf3596c-fa5f-4cc1-a0ce-6d20a221fc47 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:39.500221 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-dbf3596c-fa5f-4cc1-a0ce-6d20a221fc47 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.006949 Jun 10 13:48:39.500221 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1755/2669] 10.222.0.25 () {60 vars in 1155 bytes} [Fri Jun 10 13:48:39 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 8 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jun 10 13:48:39.505963 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a828c817-b253-4e3f-982f-43fd2ce15341 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Action: 'create', calling method: .version_select at 0x7f93099219d0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-424205188", "metadata": {"Type": "work"}}} {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a828c817-b253-4e3f-982f-43fd2ce15341 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Request method failure captured: Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 118 Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0u3JYkGwHkg4Rvh8qshGIAEBVyvEoRVNULJfdTIofcDwVy5b95KZtTtxufNN-I6YkJrYLJy0TK2puGHPkh9PG0l8m0Zs6moScv-yjXK0vdyc2oRt3sC3_sd-YXc1kWX8-f8AP4tSB_1vVfbybGqGatpfc59wUj2ffRdMvYQNtu1gsw Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: c8b90db7b2fa44269a94c63ea2512159 Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:39.508064 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:39.509442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_c8b90db7b2fa44269a94c63ea2512159", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:39.509442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.509442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: c8b90db7b2fa44269a94c63ea2512159 Jun 10 13:48:39.509442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.509442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesNegativeTest-1467618453-project Jun 10 13:48:39.509442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:39.509442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:39.509442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 9af12e2d68674326a7f217252b583a98 Jun 10 13:48:39.509442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesNegativeTest-1467618453-project Jun 10 13:48:39.509442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Jun 10 13:48:39.509442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-424205188", "metadata": {"Type": "work"}}} Jun 10 13:48:39.509442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: Jun 10 13:48:39.509442 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-424205188', 'metadata': {'Type': 'work'}}}} Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise error Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi On instance['volume']: Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi '' Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:39.510271 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jun 10 13:48:39.511677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jun 10 13:48:39.512677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jun 10 13:48:39.512677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.512677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a828c817-b253-4e3f-982f-43fd2ce15341 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:39.512677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a828c817-b253-4e3f-982f-43fd2ce15341 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.007290 Jun 10 13:48:39.512677 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1756/2670] 10.222.0.25 () {60 vars in 1155 bytes} [Fri Jun 10 13:48:39 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 8 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jun 10 13:48:39.517909 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-77cc5533-f2d3-47a6-b99b-301142353e17 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Calling method '.version_select at 0x7f93099fbc10>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:39.545129 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-77cc5533-f2d3-47a6-b99b-301142353e17 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] HTTP exception thrown: Volume tempest-invalid-1844806959 could not be found. Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-77cc5533-f2d3-47a6-b99b-301142353e17 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Request method failure captured: Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1844806959 HTTP/1.1 Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0u3JYkGwHkg4Rvh8qshGIAEBVyvEoRVNULJfdTIofcDwVy5b95KZtTtxufNN-I6YkJrYLJy0TK2puGHPkh9PG0l8m0Zs6moScv-yjXK0vdyc2oRt3sC3_sd-YXc1kWX8-f8AP4tSB_1vVfbybGqGatpfc59wUj2ffRdMvYQNtu1gsw Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: c8b90db7b2fa44269a94c63ea2512159 Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:39.547029 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:39.548398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_c8b90db7b2fa44269a94c63ea2512159", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:39.548398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.548398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: c8b90db7b2fa44269a94c63ea2512159 Jun 10 13:48:39.548398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.548398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesNegativeTest-1467618453-project Jun 10 13:48:39.548398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:39.548398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:39.548398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 9af12e2d68674326a7f217252b583a98 Jun 10 13:48:39.548398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesNegativeTest-1467618453-project Jun 10 13:48:39.548398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: Jun 10 13:48:39.548398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Volume tempest-invalid-1844806959 could not be found. Jun 10 13:48:39.548398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'tempest-invalid-1844806959'} Jun 10 13:48:39.548398 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1844806959 could not be found. (HTTP 404) (Request-ID: req-c37c027f-1625-4fa1-844d-1ee1bb460a9c) Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.549259 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jun 10 13:48:39.550437 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1844806959 could not be found. Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:48:39.551583 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1844806959 could not be found. Jun 10 13:48:39.552713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.552713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.552713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.552713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.552713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:39.552713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:39.552713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:48:39.552713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:48:39.552713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1844806959 could not be found. Jun 10 13:48:39.552713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.552713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-77cc5533-f2d3-47a6-b99b-301142353e17 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Returning 404 to user: Volume tempest-invalid-1844806959 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:39.552713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-77cc5533-f2d3-47a6-b99b-301142353e17 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1844806959" status: 404 len: 99 microversion: 2.1 time: 0.034114 Jun 10 13:48:39.552713 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1757/2671] 10.222.0.25 () {58 vars in 1217 bytes} [Fri Jun 10 13:48:39 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1844806959 => generated 99 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Jun 10 13:48:39.558606 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-de6cbb32-d3b4-4bcb-b773-8a4cb766bc0d tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004287 Jun 10 13:48:39.559104 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1758/2672] 10.222.0.25 () {58 vars in 1139 bytes} [Fri Jun 10 13:48:39 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Jun 10 13:48:39.566594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-7e63fda0-cc83-4341-b14e-7daf93b56115 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003177 Jun 10 13:48:39.567078 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1759/2673] 10.222.0.25 () {58 vars in 1136 bytes} [Fri Jun 10 13:48:39 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 4 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Jun 10 13:48:39.574512 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-21216858-92cd-410c-b214-56f9a441caf0 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Calling method '.version_select at 0x7f93099fbaf0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:39.602283 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-21216858-92cd-410c-b214-56f9a441caf0 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] HTTP exception thrown: Volume 88dfeb9f-a427-4316-b9b7-4d97b24e75f4 could not be found. Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-21216858-92cd-410c-b214-56f9a441caf0 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Request method failure captured: Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: DELETE /compute/v2.1/os-volumes/88dfeb9f-a427-4316-b9b7-4d97b24e75f4 HTTP/1.1 Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0u3JYkGwHkg4Rvh8qshGIAEBVyvEoRVNULJfdTIofcDwVy5b95KZtTtxufNN-I6YkJrYLJy0TK2puGHPkh9PG0l8m0Zs6moScv-yjXK0vdyc2oRt3sC3_sd-YXc1kWX8-f8AP4tSB_1vVfbybGqGatpfc59wUj2ffRdMvYQNtu1gsw Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: c8b90db7b2fa44269a94c63ea2512159 Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:39.604224 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:39.605602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_c8b90db7b2fa44269a94c63ea2512159", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:39.605602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.605602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: c8b90db7b2fa44269a94c63ea2512159 Jun 10 13:48:39.605602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.605602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesNegativeTest-1467618453-project Jun 10 13:48:39.605602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:39.605602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:39.605602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 9af12e2d68674326a7f217252b583a98 Jun 10 13:48:39.605602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesNegativeTest-1467618453-project Jun 10 13:48:39.605602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: Jun 10 13:48:39.605602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Volume 88dfeb9f-a427-4316-b9b7-4d97b24e75f4 could not be found. Jun 10 13:48:39.605602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': '88dfeb9f-a427-4316-b9b7-4d97b24e75f4'} Jun 10 13:48:39.605602 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 88dfeb9f-a427-4316-b9b7-4d97b24e75f4 could not be found. (HTTP 404) (Request-ID: req-a3d2bca4-3de6-4701-97ed-669bcbb8e594) Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.606426 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jun 10 13:48:39.607594 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 88dfeb9f-a427-4316-b9b7-4d97b24e75f4 could not be found. Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jun 10 13:48:39.608701 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:48:39.609806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 88dfeb9f-a427-4316-b9b7-4d97b24e75f4 could not be found. Jun 10 13:48:39.609806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.609806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.609806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.609806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.609806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:39.609806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:39.609806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:48:39.609806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:48:39.609806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 88dfeb9f-a427-4316-b9b7-4d97b24e75f4 could not be found. Jun 10 13:48:39.609806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.609806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-21216858-92cd-410c-b214-56f9a441caf0 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Returning 404 to user: Volume 88dfeb9f-a427-4316-b9b7-4d97b24e75f4 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:39.609806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-21216858-92cd-410c-b214-56f9a441caf0 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/88dfeb9f-a427-4316-b9b7-4d97b24e75f4" status: 404 len: 109 microversion: 2.1 time: 0.034336 Jun 10 13:48:39.609806 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1760/2674] 10.222.0.25 () {58 vars in 1247 bytes} [Fri Jun 10 13:48:39 2022] DELETE /compute/v2.1/os-volumes/88dfeb9f-a427-4316-b9b7-4d97b24e75f4 => generated 109 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jun 10 13:48:39.614804 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a7268eb0-a988-40ff-a2a3-fc8d9161f605 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Calling method '.version_select at 0x7f9309921ca0>' {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jun 10 13:48:39.642395 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.wsgi [None req-a7268eb0-a988-40ff-a2a3-fc8d9161f605 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] HTTP exception thrown: Volume c6943ef9-92c3-41c0-a4ae-45d4660f1566 could not be found. Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a7268eb0-a988-40ff-a2a3-fc8d9161f605 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Request method failure captured: Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: request: GET /compute/v2.1/os-volumes/c6943ef9-92c3-41c0-a4ae-45d4660f1566 HTTP/1.1 Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept: application/json Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Accept-Encoding: identity Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Connection: close Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Length: 0 Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Content-Type: application/json Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Host: 10.222.0.25 Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Openstack-System-Scope: None Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: User-Agent: python-urllib3/1.26.9 Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Auth-Token: gAAAAABio0u3JYkGwHkg4Rvh8qshGIAEBVyvEoRVNULJfdTIofcDwVy5b95KZtTtxufNN-I6YkJrYLJy0TK2puGHPkh9PG0l8m0Zs6moScv-yjXK0vdyc2oRt3sC3_sd-YXc1kWX8-f8AP4tSB_1vVfbybGqGatpfc59wUj2ffRdMvYQNtu1gsw Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Id: None Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Domain-Name: None Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Identity-Status: Confirmed Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Is-Admin-Project: True Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Id: default Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Domain-Name: Default Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Id: c8b90db7b2fa44269a94c63ea2512159 Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Project-Name: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Role: reader,member Jun 10 13:48:39.644296 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Roles: reader,member Jun 10 13:48:39.645705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_c8b90db7b2fa44269a94c63ea2512159", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c8b90db7b2fa44269a94c63ea2512159"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}] Jun 10 13:48:39.645705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.645705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Id: c8b90db7b2fa44269a94c63ea2512159 Jun 10 13:48:39.645705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-Tenant-Name: tempest-VolumesNegativeTest-1467618453 Jun 10 13:48:39.645705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User: tempest-VolumesNegativeTest-1467618453-project Jun 10 13:48:39.645705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Id: default Jun 10 13:48:39.645705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Domain-Name: Default Jun 10 13:48:39.645705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Id: 9af12e2d68674326a7f217252b583a98 Jun 10 13:48:39.645705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: X-User-Name: tempest-VolumesNegativeTest-1467618453-project Jun 10 13:48:39.645705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: method: Jun 10 13:48:39.645705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: exception: Volume c6943ef9-92c3-41c0-a4ae-45d4660f1566 could not be found. Jun 10 13:48:39.645705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: action_args: {'id': 'c6943ef9-92c3-41c0-a4ae-45d4660f1566'} Jun 10 13:48:39.645705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: {{(pid=118092) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi item = cinderclient( Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume c6943ef9-92c3-41c0-a4ae-45d4660f1566 could not be found. (HTTP 404) (Request-ID: req-2307178f-0878-4efc-a603-ec061bf1a7ce) Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.646542 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi item = cinderclient( Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jun 10 13:48:39.647705 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume c6943ef9-92c3-41c0-a4ae-45d4660f1566 could not be found. Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jun 10 13:48:39.648757 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jun 10 13:48:39.649897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume c6943ef9-92c3-41c0-a4ae-45d4660f1566 could not be found. Jun 10 13:48:39.649897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.649897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jun 10 13:48:39.649897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.649897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jun 10 13:48:39.649897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jun 10 13:48:39.649897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jun 10 13:48:39.649897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jun 10 13:48:39.649897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jun 10 13:48:39.649897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume c6943ef9-92c3-41c0-a4ae-45d4660f1566 could not be found. Jun 10 13:48:39.649897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: ERROR nova.api.openstack.wsgi Jun 10 13:48:39.649897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: DEBUG nova.api.openstack.wsgi [None req-a7268eb0-a988-40ff-a2a3-fc8d9161f605 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] Returning 404 to user: Volume c6943ef9-92c3-41c0-a4ae-45d4660f1566 could not be found. {{(pid=118092) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jun 10 13:48:39.649897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO nova.api.openstack.requestlog [None req-a7268eb0-a988-40ff-a2a3-fc8d9161f605 tempest-VolumesNegativeTest-1467618453 tempest-VolumesNegativeTest-1467618453-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/c6943ef9-92c3-41c0-a4ae-45d4660f1566" status: 404 len: 109 microversion: 2.1 time: 0.034497 Jun 10 13:48:39.649897 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: [pid: 118092|app: 0|req: 1761/2675] 10.222.0.25 () {58 vars in 1244 bytes} [Fri Jun 10 13:48:39 2022] GET /compute/v2.1/os-volumes/c6943ef9-92c3-41c0-a4ae-45d4660f1566 => generated 109 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jun 10 13:50:14.237167 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:50:14.237836 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:50:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:51:24.423613 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:51:24.424284 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:51:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:53:34.245108 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:53:34.245802 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:53:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:53:55.270968 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:53:55.271592 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:53:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jun 10 13:54:44.431912 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jun 10 13:54:44.432493 dv-cinder-master-3-iscsi devstack@n-api.service[118092]: Fri Jun 10 13:54:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!